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Chapter 1 

Introduction 


1 .1 Overview of the PIRLS 201 1 User Guide and International Database 


PIRLS provides international comparative data about students’ reading achievement after four years 
of primary schooling, while also measuring trends over time, monitoring curricular implementation, 
and identifying promising instructional practices. 1 Conducted on a regular 5-year cycle, PIRLS has 
assessed reading achievement in 2001, 2006, and 2011. PIRLS, along with its sister study prePIRLS 
introduced in 201 1, collects a rich array of background information to provide comparative 
perspectives on trends in reading achievement in the context of different educational systems, school 
organizational approaches, and instructional practices. 

As a new initiative in 201 1, IEA has developed the prePIRLS assessment in response to the 
needs of the growing population of countries requiring the kind of information provided by PIRLS, 
but whose students are earlier in the process of learning to read than is expected by the PIRLS 
assessment. The prePIRLS assessment has shorter and easier reading texts than PIRLS, and places 
less emphasis on higher-order reading skills. Three countries participated in the 2011 launch of 
prePIRLS. 

To support and promote secondary analyses aimed at improving reading education in the primary 
grades, the PIRLS 2011 International Database makes available to researchers, analysts, and other 
users the data collected and processed by the PIRLS project, as well as the data from the prePIRLS 
assessment. This database comprises student achievement data as well as student, home, teacher, 
school, and curricular background data for 48 countries and 9 benchmarking participants. The 
database includes data from 334,446 students, 281,078 parents, 15,517 teachers, 11,449 school 
principals, and the National Research Coordinators of each country. All participating countries gave 
the IEA permission to release their national data. 

For countries that participated in previous assessments, PIRLS 2011 (the third data collection in 
the PIRLS cycle of studies) provides trends for up to three cycles at the fourth grade — 2001, 2006, 
and 201 1. In countries new to the study, as well as the countries participating in the prePIRLS 
assessment, the 201 1 results can help policy makers and practitioners assess their comparative 
standing and gauge the rigor and effectiveness of their reading programs. Details of the PIRLS and 
prePIRLS assessments conducted in 201 1 can be found in PIRLS 2011 International Results in 
Reading (Mullis, Martin, Foy, & Drucker, 2012). 

PIRLS 201 1 was an ambitious and demanding study, involving complex procedures for drawing 
student samples, assessing students’ reading achievement, analyzing the data, and reporting the 

1 The Progress in International Reading Literacy Study is a project of the International Association for the Evaluation of Educational Achievement 
(IEA); PIRLS together with TIMSS, the Trends in International Mathematics and Science Study, comprise IEA’s core cycle of studies. 
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results. In order to work effectively with the PIRLS and prePIRLS data, it is necessary to have an 
understanding of the characteristics of the study, which are described fully in Methods and 
Procedures in TIMSS and PIRLS 2011 (Martin & Mullis, 2012). It is intended, therefore, that this 
User Guide be used in conjunction with the Methods and Procedures documentation. Whereas the 
User Guide describes the organization and content of the database, the Methods and Procedures 
documentation provides the rationale for the techniques used and for the variables created in the 
process of data collection and compilation. 

1 .2 Overview of the PIRLS 201 1 User Guide and International Database 

This User Guide describes the content and format of the data in the PIRLS 201 1 International 
Database. In addition to this introduction, the User Guide includes the following three chapters: 

• Chapter 2 — This chapter introduces the IEA International Database (IDB) Analyzer software 
(IEA, 2012) and presents examples of analyses with the PIRLS 2011 data using this software in 
conjunction with SPSS (IBM Corporation, 2012). 

• Chapter 3 — This chapter presents examples of analyses with the PIRLS 2011 data using the SAS 
(SAS Institute, 2011) statistical software system and the SAS programs and macros included with 
the database. 

• Chapter 4 — This chapter describes the structure and content of the database. 

The User Guide is accompanied by the following four supplements: 

• Supplement 1 — This supplement comprises the international version of all PIRLS 2011 
background questionnaires. 

• Supplement 2 — This supplement describes any adaptations to the questions in the background 
questionnaires made by individual PIRLS and prePIRLS participants. 

• Supplement 3 — This supplement describes how derived variables were constructed for reporting 
the PIRLS data. 

• Supplement 4 — This supplement describes the sampling stratification variables for each country. 

The User Guide and its four supplements are available on the PIRLS 201 1 International Database 
and User Guide webpage: http://timssandpirls.bc.edu/pirls201 1/international-database.html. 

1 .3 Contents of the PIRLS 201 1 International Database 

The entire PIRLS 2011 International Database is available on the PIRLS 2011 International Database 
and User Guide webpage: http://timssandpirls.bc.edu/pirls2011/international-database.html. The 
Database contains the PIRLS and prePIRLS 201 1 student achievement data files and student, home, 
teacher, and school background questionnaire data files, along with support materials. Exhibit 1.1 
displays the general structure of the International Database and a brief description of the support 
materials available for download on the International Database and User Guide webpage. 
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The PIRLS 2011 International Database also is available for download at the IEA Study Data 
Repository website: http://rms.iea-dpc.org/. The repository allows users to download subsets of fdes 
and the corresponding support materials through customizable queries from all recent IEA studies, 
including PIRLS and prePIRLS 2011. 


Exhibit 1 .1 : Contents of the PIRLS 2011 International Database 


User Guide 

This User Guide with its four supplements 

Items 

The PIRLS and prePIRLS 201 1 released passages and items, item information files, and IRT item parameters 

International Database 

SPSS Data 

PIRLS and prePIRLS 201 1 student, home, teacher, and school data files in SPSS format 


SAS Data 

PIRLS and prePIRLS 201 1 student, home, teacher, and school data files in SAS format 


Curriculum 

PIRLS 201 1 curriculum questionnaires data files 


Codebooks 

Codebook files describing all variables in the PIRLS 201 1 1nternational Database 


Almanacs 

Data almanacs with summary statistics for all PIRLS and prePIRLS 201 1 items and background variables 


Programs 

SAS and SPSS programs and macros 
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Chapter 2 

Using the IEA IDB Analyzer to Analyze the 
PIRLS 201 1 International Database 


2.1 Overview 


This chapter describes the use of the IEA International Database (IDB) Analyzer software (IEA, 
2012) for analyzing the PIRLS 2011 data. Used in conjunction with SPSS (IBM Corporation, 2012), 
the IEA IDB Analyzer provides a user-friendly interface to easily merge the various data file types of 
the PIRLS 201 1 database and seamlessly takes into account the sampling information and the 
multiple imputed achievement scores to produce accurate statistical results. 

Example analyses will illustrate the capabilities of the IEA IDB Analyzer (version 3.0) to 
compute a variety of statistics, including percentages of students in specified subgroups, mean 
student achievement in those subgroups, correlations, regression coefficients, percentages of students 
reaching benchmark levels, and percentiles of achievement distributions. The examples use student, 
home, teacher, and school background data to replicate some of the PIRLS 2011 results included in 
PIRLS 2011 International Results in Reading (Mullis, Martin, Foy, & Drucker, 2012). 

Users should be able to perform statistical analyses with the IEA IDB Analyzer with a basic 
knowledge of the PIRLS 2011 International Database. Chapter 4 provides a more detailed 
description of the data files contained in the International Database, including their structure and 
contents, along with a description of all the supporting documentation provided with the 
International Database. 


2.2 The IEA IDB Analyzer 

Developed by the IEA Data Processing and Research Center (IEA DPC), the IEA IDB Analyzer is a 
plug-in for SPSS, a well-known statistical analysis program. The IEA IDB Analyzer enables users to 
combine SPSS data files from IEA’s large-scale assessments and conduct analyses using SPSS 
without actually writing programming code. The IEA IDB Analyzer generates SPSS syntax that 
takes into account information from the sample design in the computation of statistics and their 
standard errors. In addition, the generated SPSS syntax makes appropriate use of plausible values for 
calculating estimates of achievement scores and their standard errors, combining both sampling 
variance and imputation variance. 

The IEA IDB Analyzer consists of two modules — the merge module and the analysis module — 
which are integrated and executed in one common application. The merge module is used to create 
analysis datasets by combining data files of different types and from different countries, and 
selecting subsets of variables for analysis. The analysis module provides procedures for computing 
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various statistics and their standard errors. The latest version (3.0) of the IEA IDB analyzer is 
available for download at the IEA website: http://www.iea.nl/data.html. Once installed, the IEA IDB 
Analyzer can be accessed by using the START menu in Windows: 

Start ■=> All Programs ■=> IEA O IDB Analyzer V3 ■=> IEA IDB Analyzer 


2.3 Merging Files with the IEA IDB Analyzer 

The IEA IDB Analyzer uses the SPSS data files available on the PIRLS 201 1 International Database 
and User Guide webpage: http://timssandpirls.bc.edu/pirls2011/international-database.html} The 
PIRLS and prePIRLS 2011 data files are disseminated separately by file type and for each country. 

In addition to allowing users to combine like datasets from more than one country for cross-country 
analyses, the merge module allows for the combination of data from different sources (e.g., student, 
home, teacher, and school) into one SPSS dataset for subsequent analyses. Before conducting any 
statistical analyses with the PIRLS 2011 International Database, users should download and copy the 
contents of the International Database either on their computer or on a server. For the purposes of 
this chapter, we will assume all files have been copied to the folder titled “C:\PIRLS201 1\.” 

The following steps will create an SPSS data file with data from multiple countries and/or 
multiple file types: 

1 . Start the IEA IDB Analyzer from the START menu and click the Merge Module button. 

2. Under the Select Data Files and Participants tab, in the Select Directory field, browse to the 
folder where all SPSS data files are located. For example, in Exhibit 2.1, all SPSS data files are 
located in the folder titled “C:\PIRLS201 l\Data\SPSS_Data.” The program will automatically 
recognize and complete the Select Study, Select Year, and Select Grade fields and list all 
countries available in this folder as possible candidates for merging. If the folder contains data 
from more than one IEA study, or from more than one grade, the IEA IDB Analyzer will prompt 
users to select files from the desired combination of study and grade for analyses. In Exhibit 2.1, 
PIRLS 2011 is selected. 

3. Click a country of interest from the Available Participants list and click the right arrow button 
( ► ) to move it to the Selected Participants panel. Individual countries can be moved directly to 
the Selected Participants panel by double-clicking on them. To select multiple countries, hold 
the CTRL key of the keyboard when clicking countries. Click the tab-right arrow button ( ►( ) to 
move all countries to the Selected Participants panel. In Exhibit 2.1, Australia, Austria, 
Azerbaijan, and the French-speaking community of Belgium are selected. 


1 The PIRLS 2011 International Database also is available for download on the IEA Study Data Repository website: http://rms.iea-dpc.org/. 
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Exhibit 2.1 : IEA IDB Analyzer Merge Module — Select Data Files and Participants 


£ IEA IDB Analyzer Merge Module - (Version 3.0.47) 

Select Data Files and Participants Select File Types and Variables 


3 Select Directory... 


I C:\PIRLS2011\Data\SPSS_Data 

_J 1 

^ 1 

Select Study: 

Select Year: 

Select Grade: 


| PIRLS 

H | PIRLS 2011 

w | Grade 4 



Q Available Participants: (54) 


Selected Participants: (4) 


[ Code 

Name 

T 


&AAD 

United Arab Emirates (Abu Dhabi) 

4? ADU 

United Arab Emirates (Dubai) 

4? ARE 

United Arab Emirates 

&BGR 

Bulgaria 

4?BWA 

Botswana 

4? CAB 

Canada (Alberta) 

4? CAN 

Canada 

4? COL 

Colombia 

4? COT 

Canada (Ontario) 

J0CQU 

Canada (Quebec) 

4?CZE 

Czech Republic 

4? DEU 

Germany 

&DNK 

Denmark 

4? EAN 

Spain (Andalucia) 

4? ENG 

England 

4? ESP 

Spain 

4? FIN 

Finland 

4? FRA 

France 

4? GEO 

Georgia 

& HKG 

Hong Kong, SAR 

4? HND 

Honduras, Republic of T 




| Next > | 


| Return to Main Menu | | Help | 


PIRLS 2011 Grade 4 


4. Click the Next > button to proceed to the next step in the Select File Types and Variables tab. 
The software will open the second tab of the merge module, as shown in Exhibit 2.2, to select the 
file types and the variables to be included in the merged data file. 

5. Select the files for merging by checking the appropriate boxes to the left of the window. For 
example, in Exhibit 2.2, the student background data files are selected. 

6. Select the variables of interest from the Available Variables list in the left panel. Note that 
Supplement 1 provides the variable names for all questions in the background questionnaires. 
Variables are selected by clicking on them and then clicking the right arrow ( ► ) button. 
Clicking the tab-right arrow ( ►( ) button selects all variables. Note that there are two tabs: 
“Background Variables and Scores” and “ID and Sampling Variables.” All achievement scores 
and all identification and sampling variables are selected automatically by the IEA IDB 
Analyzer. 
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Exhibit 2.2: IEA IDB Analyzer Merge Module — Select File Types and Variables 



7. Specify the desired name for the merged data file and the folder where it will be stored in the 
Output Files field by clicking the Define/Modify button. The IEA IDB Analyzer also will create 
an SPSS syntax file (*.SPS) of the same name and in the same folder with the code necessary to 
perform the merge. In the example shown in Exhibit 2.2, the merged file ASGALLR3.SAV and 
the syntax file ASGALLR3.SPS both will be created and stored in the folder titled 
“C:\PIRLS201 lYData.” The merged data file will contain all the variables listed in the Selected 
Variables panel to the right. 

8. Click the Start SPSS button to create the SPSS syntax file and open it in an SPSS syntax 
window ready for execution. The syntax file can be executed by opening the Run menu of SPSS 
and selecting the All menu option. The IEA IDB Analyzer will display a warning if it is about to 
overwrite an existing file in the specified folder. 

Once SPSS has completed its execution, it is important to check the SPSS output window for 
possible warnings. If warnings appear, they should be examined carefully because they might 
indicate that the merge process was not performed properly and that the resulting merged data file 
might not be as expected. 
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Merging Student and Teacher Data Files 

The teachers in the PIRLS 201 1 International Database do not constitute representative samples of 
teachers in the participating countries. Rather, they are the teachers of nationally representative 
samples of students. Therefore, analyses with teacher data should be made with students as the units 
of analysis and reported in terms of students who are taught by teachers with a particular attribute. 

Teacher data are analyzed by linking the students to their teachers. The student-teacher linkage 
data files (AST) are used for this purpose and the IEA IDB Analyzer will make use of them 
automatically. Thus, to analyze teacher data, it is sufficient to select the Teacher Background file 
types in the Select File Types and Variables tab of the IEA IDB Analyzer merge module. To 
analyze student and teacher background data simultaneously, however, both the Student 
Background and Teacher Background file types must be selected in the Select File Types and 
Variables tab of the IEA IDB Analyzer merge module. The variables of interest need to be selected 
separately for both file types, as follows: 

1. Click the Teacher Background file type so that it appears checked and highlighted. The 
Background Variables and Scores listed in the left panel will include all available variables 
from the teachr background data files. 

2. Select the variables of interest from the left panel and click the right arrow ( ► ) button to move 
these variables to the Selected Variables panel on the right. Click the tab-right arrow ( ►( ) 
button to select all available variables. 

3. If necessary, click the Student Background file type and selecting the variables of interest from 
the Background Variables and Scores panel on the left in the same manner. 

4. Specify the folder and merged data file name in the Output Files field, as described earlier. 

5. Click the Start SPSS button to create the SPSS syntax file that will produce the required merged 
data file, which can then be run by opening the Run menu of SPSS and selecting the All menu 
option. 

Merging Student and School Data Files 

Because PIRLS 2011 has representative samples of schools, it is possible to compute reasonable 
statistics with schools as units of analysis. However, the school samples were designed to optimize 
the student samples and the student-level estimates. For this reason, it is preferable to analyze 
school-level variables as attributes of the students, rather than as elements in their own right. 
Therefore, analyzing school data should be done by linking the students to their schools. 

To merge the student and school background data files, select both the Student Background and 
School Background file types in the Select File Types and Variables tab of the IEA IDB Analyzer 
merge module. The variables of interest to be included in the merged data file are selected separately 
by file type, as described above in “Merging Student and Teacher Data Files” and using the same set 
of instructions. 
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Merging Student and Home Data Files 

The parents of the students participating in PIRLS and prePIRLS 2011 responded to a Learning to 
Read survey, usually called the Home Questionnaire. Their responses are included in the home 
background data files (ASH). Although home background variables are located in separate files, they 
are in essence attributes of students and must be analyzed in the same manner as student background 
variables. This will require users to merge the home background data files with the student 
background data files by selecting both the Home Background and Student Background file types. 
Variables of interest to be included in the merged data file must be chosen separately by file type, as 
described above in “Merging Student and Teacher Data Files” and using the same set of instructions. 

Merged Data Files for the Examples 

To conduct the analysis examples presented in this chapter, the following merged data files should be 
created with all available background variables and achievement scores selected: 

ASGALLR3.SAV — Merged PIRLS student background data files for all countries. 

ASHALLR3.SAV — Merged PIRLS home and student background data files for all countries. 

ATGALLR3.SAV — Merged PIRLS teacher background data files for all countries. 

ACGALLR3.SAV — Merged PIRLS school and student background data files for all countries. 

2.4 Performing Analyses with the IEA IDB Analyzer 

The analysis module of the IEA IDB Analyzer can perform statistical analyses on any files created 
using the merge module. The following statistical procedures are available in the analysis module of 
the IEA IDB Analyzer. 

Percentages and Means 

Compute percentages, means, and standard deviations for selected variables by subgroups defined by 
grouping variable(s). 

Percentages Only 

Compute percentages by subgroups defined by grouping variable(s). 

Regression 

Compute regression coefficients for selected independent variables to predict a dependent variable 
by subgroups defined by grouping variable(s). Plausible values can be included as dependent or 
independent variables. 

Correlations 

Compute means, standard deviations, and correlation coefficients for selected variables by subgroups 
defined by grouping variable(s). 
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Benchmarks 


Compute percentages of students meeting a set of user-specified achievement benchmarks, in 
particular the PIRLS International Benchmarks, by subgroups defined by grouping variable(s). 

Percentiles 

Compute the score points that separate a given proportion of the distribution of achievement scores, 
by subgroups defined by the grouping variable(s). 

All statistical procedures offered in the analysis module of the IEA IDB Analyzer make 
appropriate use of sampling weights, and standard errors are computed using the jackknife repeated 
replication (JRR) method . 2 Percentages, means, regressions, and correlations may be specified with 
or without achievement scores. When achievement scores are used, the analyses are performed five 
times (once for each plausible value) and the results are aggregated to produce accurate estimates of 
achievement and standard errors that incorporate both sampling and imputation errors. To conduct 
analyses using achievement scores, select the Use PVs option from the Plausible Value Option 
drop-down menu. The various variables required to perform an analysis are input into specific 
variable fields according to their purpose. 

Grouping Variables 

This is a list of variables to define subgroups. The list must consist of at least one grouping variable. 
By default, the IEA IDB Analyzer includes the variable IDCNTRY used to distinguish the 
participating countries. Additional variables may be selected from the available list. If the 
Exclude Missing from Analysis option is checked, only cases that have non-missing values in the 
grouping variables will be used in the analysis. If it is not checked, missing values become reporting 
categories. 

Analysis Variables 

This is a list of variables for which means or percentages are to be computed. More than one analysis 
variable can be selected. To compute means for achievement scores, it is necessary to select the 
Use PVs option in the Plausible Value Option drop-down menu, and select the achievement scores 
of interest in the Plausible Values field. 

Plausible Values 

This section is used to identify the set of plausible values to be used when achievement scores are the 
analysis variable for computing percentages and means. Select the Use PVs option in the Plausible 
Value Option drop-down menu before specifying the achievement scores in the Plausible Values 
field. 


2 See Methods and Procedures in TIMSS and PIRLS 2011 (Martin & Mullis, 2012). 
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Independent Variables 


This is a list of variables to be treated as independent variables for a regression analysis. More than 
one independent variable can be selected. Achievement scores also can be included as an 
independent variable. To specify achievement scores as the independent variable, it is necessary to 
select the Use PVs option in the Plausible Value Option drop-down menu and select the 
achievement scores of interest in the Plausible Values field. 

Dependent Variable 

This is the variable to be used as the dependent variable when a regression analysis is specified. Only 
one dependent variable can be listed and can be either a background variable or the set of five scores 
associated with an achievement scale. To use achievement scores as the dependent variable, it is 
necessary to select the Use PVs option in the Plausible Value Option drop-down menu and select 
the achievement scores of interest in the Plausible Values field. 

Weight Variable 

This is the sampling weight variable that will be used in the analysis. The IE A IDB Analyzer 
automatically selects the appropriate weight variable for analysis based on the file types included in 
the merged data file. Generally, this will be TOTWGT, but SENWGT and HOUWGT also are 
available for student-level analyses with student, home, or school data. TCHWGT will be used when 
analyzing teacher data. Chapter 4 provides more information on the PIRLS sampling weights. 

Achievement Benchmarks 

These are the values that will be used as cut points on an achievement scale, selected in the Plausible 
Values section, for computing the percentages of students meeting the specified benchmarks. 
Multiple cut points can be specified, each separated by a blank space. It is necessary to select the Use 
PVs option in the Plausible Value Option drop-down menu and select the achievement scores of 
interest in the Plausible Values field. 

Percentiles 

These are the percentiles that will be calculated from the distribution of the achievement scores 
selected in the Plausible Values section. Multiple percentiles can be specified, each separated by a 
blank space. It is necessary to select the Use PVs option in the Plausible Value Option drop-down 
menu and select the achievement scores of interest in the Plausible Values field. 
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2.5 PIRLS/prePIRLS Analyses with Student-level Data 


Many analyses of the PIRLS 2011 International Database can be undertaken using only student-level 
data. This section presents examples of actual analyses used to produce exhibits from PIRLS 2011 
International Results in Reading. Examples of regression analyses and computing percentages of 
students reaching the PIRLS International Benchmarks also are included in this section. 

A first example computes national average achievement, whereas a second example computes 
national average achievement by gender. In both cases, the IEA IDB Analyzer uses the sampling 
weights, implements the jackknife repeated replication method to compute appropriate sampling 
errors, effectively performs the computations five times (once for each plausible value), and 
aggregates the results to produce accurate estimates of average achievement and standard errors that 
incorporate both sampling and imputation errors. A third example expands on the second example by 
performing a test of significance on the gender difference using regression. A fourth example 
computes the percentages of students reaching the PIRLS International Benchmarks. 

Student-level Analysis with Achievement 

In our first example, we want to replicate the analysis of the overall distribution of reading 
achievement. These results are presented in Exhibit 1.1 of PIRLS 2011 International Results in 
Reading and are repeated here in Exhibit 2.3. Because the results in this exhibit are based on 
plausible values, we must ensure that we include them when we create the file using the merge 
module, and also indicate that our analysis will make use of achievement scores. 

The Percentages and Means statistic type with the Use PVs option selected will compute 
percentages and average reading achievement based on plausible values and their respective standard 
errors. After creating the merged data file ASGALLR3, the analysis module of the IEA IDB 
Analyzer is used to perform the analysis using the following steps: 

1 . Open the Analysis Module of the IEA IDB Analyzer. 

2. Select the merged data file ASGALLR3 as the Analysis File by clicking the Select button. 

3. Select PIRLS (Using Student Weights) as the Analysis Type. 

4. Select Percentages and Means as the Statistic Type. 

5. Select Use PVs as the Plausible Value Option. 

6. The variable IDCNTRY is selected automatically as Grouping Variables. No additional 
grouping variables are needed for this analysis. 

7. Specify the achievement scores to be used for the analysis and activate them by clicking the 
Plausible Values field. Select ASRREA01-05 from the list of available variables and move it to 
the Plausible Values field by clicking the right arrow ( ► ) button in this section. 

8. The Weight Variable is selected automatically by the software; TOTWGT is selected by default 
because this example analysis uses student background data. 

9. Specify the name and the folder of the output files in the Output Files field by clicking the 
Define/Modify button. 
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Exhibit 2.3: Exhibit of Example Student-level Analysis with Achievement, Taken from 

PIRLS 2011 International Results in Reading (Exhibit 1 .1 ) 



100 200 300 400 


500 600 


Country 


Average 
Scale Score 


Reading Achievement Distribution 


Sixth Grade Participants 

Honduras 450 (4.8) 

Morocco 424 (3.9) 

i * Kuwait 419 (5.2) 

Botswana 419 (4.1) 


Benchmarking Participants 0 


i 3 Florida, US 569 (2.9) 

2 Ontario, Canada 552 (2.6) 

2 Alberta, Canada 548 (2.9) 

Quebec, Canada 538 (2.1) 

Andalusia, Spain 515 (2.3) 

Dubai, UAE 476 (2.0) 

Maltese - Malta 457 (1.5) 

Abu Dhabi, UAE 424 (4.7) 

vEng/Afr(5)-RSA 421 (7.3) 

0 Republic of South Africa (RSA) tested 5th grade students receiving 
instruction in English (ENG) or Afrikaans (AFR). 


® 

® 

® 

® 




O 

o 

M . — 

® — 

* 

® — 

I 1 1 

100 200 300 

O Country average significantly higher than 
the centerpoint of the PIRLS scale 
® Country average significantly lower than 
the centerpoint of the PIRLS scale 


-i 1 r 

500 600 700 

Percentiles of Performance 


5th 25th 75th 95th 



95% Confidence Interval for Average (±2SE) 


M Average achievement not reliably measured because the percentage of students with achievement too low for estimation exceeds 25%. 

V Reservations about reliability of average achievement because the percentage of students with achievement too low for estimation does not exceed 25% but exceeds 1 5%. 
See Appendix C.2 for target population coverage notes 1, 2, and 3. See Appendix C.5 for sampling guidelines and sampling participation notes t and 4. 

( ) Standard errors appear in parentheses. Because of rounding some results may appear inconsistent 


14 


PIRLS 2011 USER GUIDE FOR THE INTERNATIONAL DATABASE 

CHAPTER 2 


TIMSS & PIRLS 

International Study Center 

Lynch School of Education, Boston College 





10. Click the Start SPSS button to create the SPSS syntax file and open it in an SPSS syntax 
window. The syntax file can be executed by opening the Run menu of SPSS and selecting the 
All menu option. If necessary, the IEA IDB Analyzer will display a prompt to confirm the 
overwriting of existing files. 

Exhibit 2.4 shows the completed analysis module for this example analysis, and Exhibit 2.5 
displays the results with our four example countries. 

Exhibit 2.4: IEA IDB Analyzer Setup for Example Student-level Analysis with Achievement 


F IEA IDB Analyzer Analysis Module - (Version 3.0.47) 




O Analysis File: | C:\PIRLS2011\Data\ASGALLR3.sav [ Select ] 


a 





Analysis Type: 

Statistic Type: 

Plausible Value Option: Benchmark Option: 

Missing Data Option: Number of Decimals: 


| PIRLS (Using Student Weights) 

w | | Percentages and Means 

| [Use PVs H | 

HU h 








3 Select Variables: 




O Output Files: C:\PIRLS2011\Data\REAOverall* 


| Modify | 
O [ Start SPSS | 


| Return to Main Menu 


Help | 


PIRLS (Using Student Weights) Percentages and Means Use PVs 


Exhibit 2.5: Output for Example Student-level Analysis with Achievement 


Average for ASRREAO by IDCNTRY 


* COUNTRY ID* 

N of 
Cases 

Sum of 
TOTWGT 

Percent 

Percent 
(s.e. ) 

ASRREAO 

(Mean) 

ASRREAO 

(s.e.) 

Std.Dev 

Std.Dev 

(s.e.) 

Australia 

6126 

251213 

1.24 

.03 

527.37 

2.21 

80.22 

1.31 

Austria 

4670 

78333 

.39 

. 01 

528 . 88 

1 . 95 

63.38 

. 95 

Azerbaijan, Republic of 

4881 

104014 

.51 

. 02 

462.30 

3.33 

67 . 83 

1 . 68 

Belgium (French) 

3727 

49879 

.25 

.01 

506.12 

2.88 

64.67 

1.57 


Each country’s results are presented on a single line. The countries are identified in the first 
column and the second column reports the number of valid cases. The third column identifies the 
sum of weights of the sampled students. The next four columns report the percentage of students in 
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each category (Country) and its standard error, followed by the estimated average reading 
achievement and its standard error. The last two columns report the standard deviation of the 
achievement scores and its standard error. 

As shown in the first line in Exhibit 2.5, Australia had valid data for 6,126 students and these 
sampled students represented a population of 251,213 students. The average reading achievement in 
Australia was 527.37 (standard error of 2.21), and its standard deviation was 80.22 (standard error 
of 1.31). 

Student-level Analysis with Achievement by Gender 

In our second example, we want to replicate another set of results presented in PIRLS 2011 
International Results in Reading. We are interested in investigating the relationship between student 
gender and reading achievement. These results, presented in Exhibit 1.5 of PIRLS 2011 International 
Results in Reading, are repeated here in Exhibit 2.6. Because the results in this exhibit are based on 
plausible values, we must ensure that these values are included when creating the input file, and also 
indicate that this analysis will make use of achievement scores. 

After reviewing the appropriate codebook, the variable ITSEX contains categorical information 
on the gender of students, and this variable is found in the student background data files. The 
Percentages and Means statistic type and the Use PVs plausible value option will compute the 
percentages and average achievement based on plausible values and their respective standard errors. 
The analysis module of the IEA IDB Analyzer is used to perform the analysis using the following 
steps: 

1 . Open the Analysis Module of the IEA IDB Analyzer. 

2. Select the merged data file ASGALLR3 as the Analysis File by clicking the Select button. 

3. Select PIRLS (Using Student Weights) as the Analysis Type. 

4. Select Percentages and Means as the Statistic Type. 

5. Select Use PVs as the Plausible Value Option. 

6. Specify the variable ITSEX as a second grouping variable and activate it by clicking the 
Grouping Variables field. Select ITSEX from the list of available variables and move it to the 
Grouping Variables field by clicking the right arrow ( ► ) button in this section. 

7. Specify the achievement scores to be used for the analysis and activate them by clicking the 
Plausible Values field. Select ASRREA01-05 from the list of available variables and move it to 
the Plausible Values field by clicking the right arrow ( ► ) button in this section. 

8. The Weight Variable is selected automatically by the software; TOTWGT is selected by default 
because this example analysis uses student background data. 

9. Specify the name and the folder of the output files in the Output Files field by clicking the 
Define/Modify button. 
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Exhibit 2.6: Exhibit of Example Student-level Analysis with Achievement by Gender, Taken from 

PIRLS 2011 International Results in Reading (Exhibit 1 .5) 


Exhibit 1 .5: Average Reading Achievement by Gender 


PIRLS 2011 


4 th 

Grade 


Country 


Girls 




Boys 


Difference 

(Absolute 

Value) 

Gender Difference 

Percent of 
Students 

Average Scale 
Score 

Percent of 
Students 

Average Scale 
Score 

Girls 

Scored Higher 

Boys 

Scored Higher 

Colombia 

49 (1.3) 


447 (4.6) 


51 (1.3) 


448 (4.6) 

1 (3.9) 






Italy 

50 (0.7) 

II 

543 (2.4) 


50 (0.7) 


540 (2.7) 

3 (2.4) 




i 


France 

49 (0.8) 


522 (3.4) 


51 (0.8) 


518 (2.4) 

5(2.7) 




■ 


Spain 

49 (0.8) 

II 

516 (2.5) 

II 

51 (0.8) 

■I 

511 (2.8) 

5 (2.5) 




■ 


2 1 Belgium (French) 

49 (0.9) 


509 (3.1) 


51 (0.9) 


504 (3.1) 

5(2.3) 




■ 


3 Israel 

51 (1.6) 

II 

544 (3.1) 

II 

49 (1.6) 


538 (3.4) 

6(3.4) 




■ 


Czech Republic 

49 (1.2) 


549 (2.5) 


51 (1.2) 


542 (2.5) 

6(2.6) 




■ 


f Netherlands 

51 (0.7) 

|| 

549 (2.1) 

|| 

49 (0.7) 


543 (2.2) 

7(2.0) 




■ 


Austria 

49 (1.2) 


533 (2.2) 


51 (1.2) 


525 (2.3) 

8(2.3) 




■ 


Germany 

49 (0.8) 

II 

545 (2.3) 

II 

51 (0.8) 


537 (2.7) 

8(2.5) 




■ 


Slovak Republic 

49 (0.8) 


540 (3.1) 


51 (0.8) 


530 (2.8) 

10(2.1) 




■ 


2 United States 

51 (0.5) 

|| 

562 (1.9) 

II 

49 (0.5) 


551 (1.7) 

10 (1.8) 






2 Denmark 

50 (0.7) 


560 (1.9) 


50 (0.7) 


548 (2.1) 

12(2.2) 






2 Canada 

49 (0.6) 

II 

555 (1.7) 

II 

51 (0.6) 

■I 

542 (2.1) 

12(2.0) 




■i 


Poland 

48 (0.9) 


533 (2.5) 


52 (0.9) 


519 (2.7) 

14(3.1) 






2 Azerbaijan 

47 (0.9) 

II 

470 (3.6) 

II 

53 (0.9) 

■I 

456 (3.5) 

14(2.3) 






2 Croatia 

50 (0.8) 


560 (2.1) 


50 (0.8) 


546 (2.2) 

14(2.2) 






Sweden 

49 (1.0) 

II 

549 (2.4) 

II 

51 (1.0) 

■1 

535 (2.5) 

14(2.7) 






Portugal 

49 (1.2) 


548 (3.0) 


51 (1.2) 


534 (2.8) 

14 (2.4) 






* Norway 

52 (1.0) 

II 

514 (2.2) 

II 

48 (1.0) 


500 (2.7) 

14(3.1) 






Chinese Taipei 

47 (0.6) 


561 (2.1) 


53 (0.6) 


546 (2.1) 

15(2.1) 






Bulgaria 

49 (0.9) 

II 

539 (4.5) 

II 

51 (0.9) 


524 (4.3) 

15(3.5) 






Romania 

48 (0.9) 


510 (4.8) 


52 (0.9) 


495 (4.3) 

15(3.3) 






Ireland 

49 (2.2) 

II 

559 (2.9) 

II 

51 (2.2) 


544 (3.0) 

15(3.9) 






Hungary 

49 (0.9) 


547 (3.2) 


51 (0.9) 


532 (3.2) 

16 (2.6) 






Slovenia 

48 (0.8) 

II 

539 (2.2) 

II 

52 (0.8) 

■1 

523 (2.7) 

16(3.1) 






t Northern Ireland 

50 (1.2) 


567 (2.5) 


50 (1.2) 


550 (3.2) 

16 (3.4) 






3 Hong Kong SAR 

46 (1.2) 

II 

579 (2.3) 

II 

54 (1.2) 

■I 

563 (2.5) 

16(2.2) 






Australia 

49 (1.1) 


536 (2.7) 


51 (1.1) 


519 (2.7) 

17(3.1) 




■■ 


2 Singapore 

49 (0.6) 

II 

576 (3.5) 

II 

51 (0.6) 

■I 

559 (3.6) 

17 (2.6) 






Malta 

49 (0.5) 


486 (1.9) 


51 (0.5) 


468 (2.0) 

18 (2.8) 






Indonesia 

51 (0.9) 

II 

437 (4.5) 

|| 

49 (0.9) 

■1 

419 (4.3) 

18(2.3) 






1 2 Lithuania 

48 (0.8) 


537 (2.4) 


52 (0.8) 


520 (2.4) 

18(2.8) 






Russian Federation 

49 (1.0) 

II 

578 (2.8) 

II 

51 (1.0) 

■I 

559 (3.1) 

18(2.3) 






Iran, Islamic Rep. of 

49 (2.9) 


467 (4.3) 


51 (2.9) 


448 (4.3) 

20 (6.4) 






New Zealand 

49 (1.0) 

II 

541 (2.2) 

II 

51 (1.0) 

■I 

521 (2.7) 

20(3.1) 






Finland 

49 (0.8) 


578 (2.3) 


51 (0.8) 


558 (2.2) 

21 (2.3) 






1 Georgia 

48 (0.9) 

II 

499 (2.7) 

II 

52 (0.9) 

■1 

477 (4.0) 

22 (3.0) 






t England 

49 (1.0) 


563 (3.0) 


51 (1.0) 


540 (3.1) 

23 (3.0) 




■■■■ 


United Arab Emirates 

50 (1.6) 

II 

452 (3.0) 

II 

50 (1.6) 


425 (3.5) 

27 (4.8) 






>k Morocco 

48 (0.8) 


326 (4.0) 


52 (0.8) 


296 (4.6) 

29 (3.9) 






2 Qatar 

47 (3.4) 

II 

441 (4.7) 


53 (3.4) 


411 (4.2) 

30 (6.0) 






Trinidad and Tobago 

49 (2.0) 


487 (4.5) 


51 (2.0) 


456 (4.3) 

31 (4.6) 






v Oman 

49 (0.7) 

II 

411 (3.0) 

II 

51 (0.7) 


371 (3.4) 

40 (2.9) 






Saudi Arabia 

52 (1.5) 


456 (3.1) 


48 (1.5) 


402 (8.2) 

54 (8.8) 







KTgQ] 




■MOW 










80 40 0 40 80 



Girls 

Boys 

Difference Gender Difference 

Country 

Percent of 
Students 

Average Scale 
Score 

Percent of 
Students 

Average Scale 
Score 

(Absolute Girls Boys 

Value) Scored Higher Scored Higher 

Sixth Grade Participants 

Honduras 

51 (U) 


455 (5.5) 

49(1.2) 

444 (5.0) 

12 (4.2) — 

Botswana 

51 (0.8) 


432 (4.2) 

49 (0.8) 

405 (4.8) 

28(3.4) 

Morocco 

48 (0.7) 


443 (3.8) 

52 (0.7) 

408 (4.5) 

35 (3.5) 

1 * Kuwait 

54(1.9) 


443 (6.4) 

46(1.9) 

391 (7.3) 

53 (9.3) 

Benchmarking Participants 0 

Andalusia, Spain 

50 (0.9) 


519 (2.4) 

50 (0.9) 

511 (2.8) 

8 (2.6) ■ 

2 Alberta, Canada 48 (0.9) 

■1 

553 (3.1) 

52 (0.9) 

543 (3.1) 

10(2.2) - 

2 Ontario, Canada 

49(1.1) 


558 (3.3) 

51 (1.1) 

546 (2.8) 

13 (3.4) — 

Dubai, UAE 47 (2.3) 

■1 

483 (3.9) 

53 (2.3) 

470 (3.5) 

13 (6.3) ™ 

Quebec, Canada 

50 (1.0) 


544 (2.6) 

50 (1.0) 

531 (2.4) 

14 (2.5) ™ 

i 3 Florida, US 

51 (0.9) 

■1 

576 (3.4) 

49 (0.9) 

561 (3.0) 

15 (2.9) — 

Maltese - Malta 

49 (0.5) 


470 (2.0) 

51 (0.5) 

445 (2.2) 

25 (3.0) 

y Eng/Afr (5) - RSA 

49(1.3) 


434 (7.7) 

51 (1.3) 

408 (8.7) 

26(7.7) | 

Abu Dhabi, UAE 

50 (2.9) 


442 (5.5) 

50 (2.9) 

406 (6.3) 

36 (8.0) 


0 Republic of South Africa (RSA) tested 5th grade students receiving instruction in English (ENG) or Afrikaans (AFR). 


■ Difference statistically significant 

■ Difference not statistically significant 

>K Average achievement not reliably measured because the percentage of students with achievement too low for estimation exceeds 25%. 

V Reservations about reliability of average achievement because the percentage of students with achievement too low for estimation does not exceed 25% but exceeds 1 5%. 
See Appendix C 2 for target population coverage notes 1, 2, and 3. See Appendix C.5 for sampling guidelines and sampling participation notes t and 4. 

( ) Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 
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10. Click the Start SPSS button to create the SPSS syntax file and open it in an SPSS syntax 
window. The syntax file can be executed by opening the Run menu of SPSS and selecting the 
All menu option. If necessary, the IEA IDB Analyzer will display a prompt to confirm the 
overwriting of existing files. 

Exhibit 2.7 shows the completed analysis module for this example analysis and the results are 
presented in Exhibit 2.8. 

Exhibit 2.7: IEA IDB Analyzer Setup for Example Student-level Analysis with Achievement by Gender 


F IEA IDB Analyzer Analysis Module - (Version 3.0.47) 




O Analysis File: | C:\PIRLS2011\Data\ASGALLR3.sav [ Select ] 


a 





Analysis Type: 

Statistic Type: 

Plausible Value Option: Benchmark Option: 

Missing Data Option: Number of Decimals: 


| PIRLS (Using Student Weights) 

w | | Percentages and Means 

| | Use PVs 

H|2 H 








0 Select Variables: 


A Grouping Variables: SI Exclude Missing From Analysis 



[ Name 

Description 

1 

¥ 


4$asbgscr 

•STUDENTS CONFIDENT IN THEIR READING/SCL P* 

* 

4$asdgscr 

•STUDENTS CONFIDENT IN THOR READING/IDX P* 


40ASBGERL 

•STUDENTS ENGAGED IN READING LESSONS/SCL P* 


40ASDGERL 

•STUDENTS ENGAGED IN READING LESSONS/IDX P* 


4$ASDG05S 

•NUMBER OF HOME STUDY SUPPORTS’ 


4$asdrlowp 

•READING ACH TOO LOW FOR ESTIMATION* 


<#DPCDATE 

•FILE CREATION DATE AT DPC HAMBURG* 


4 IDBOOK 

•ACHIEVEMENT TEST BOOKLET’ 


f IDSCHOOL 

•SCHOOL ID* 


4 I DC LASS 

•CLASS ID* 


j IDSTUD 

’STUDENT ID’ 


& IDGRADE 

•GRADE ID’ 


0 ITBIRTHD 

•DATE OF STUDENTS BIRTHXDAY* 


f ITBIRTHM 

•DATE OF STUDENTS BIRTH\MONTH* 


^ itbirthy 

•DATE OF STUDENTS BIRTH\YEAR’ 


f ITADMINI 

TEST ADMINISTRATORS POSITION* 

n 

/ ITDATE 

•DATE OF TESTING’ 


0 ITLANG 

•LANGUAGE OF TESTING* 


/ ITACCOMM1 

•SPECIAL ACCOMMODATION/ACH SESSIO’ 

J 

& IDPOP 

•POPULATION ID’ 

i 


O Output Files: | C:\PIRLS2011\Data\REAbyGender.* 


| Modify | 
O [ Start SPSS ] 


| Return to Main Menu 


Help | 


PIRLS (Using Student Weights) Percentages and Means Use PVs 


Exhibit 2.8: Output for Example Student-level Analysis with Achievement Scores by Gender 


Average for ASRREAO by IDCNTRY ITSEX 


* COUNTRY ID* 

*SEX OF STUDENTS* 

N of 
Cases 

Sum of 
TOTWGT 

Percent 

Percent 
(s.e. ) 

ASRREAO 

(Mean) 

ASRREAO 

(s.e.) 

Std.Dev 

std.Dev 
(s.e. ) 

Australia 

GIRL 

3048 

123760 

49.26 

1.06 

535.79 

2 . 67 

78.20 

1.62 


BOY 

3078 

127453 

50 . 74 

1 . 06 

519.20 

2 . 73 

81.30 

1 . 75 

Austria 

GIRL 

2274 

38141 

48.69 

1.18 

532.76 

2 . 18 

62.00 

1.21 


BOY 

2396 

40192 

51.31 

1 . 18 

525 . 19 

2.32 

64.44 

1.48 

Azerbaijan, Republic of 

GIRL 

2241 

49037 

47 . 14 

.85 

469.57 

3.56 

67.31 

1 . 94 


BOY 

2640 

54977 

52 . 86 

. 85 

455 . 82 

3.47 

67.63 

1.85 

Belgium (French) 

GIRL 

1815 

24388 

48.89 

.93 

508.85 

3.11 

63.11 

2 . 01 


BOY 

1912 

25491 

51 . 11 

. 93 

503.51 

3.11 

66.02 

1.62 
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Each country’s results are displayed on two lines, one for each value of the ITSEX variable. The 
countries are identified in the first column and the second column describes the category of ITSEX 
being reported. The third column reports the number of valid cases and the fourth the sum of weights 
of the sampled students. The next four columns report the percentage of students in each category 
and its standard error, followed by the estimated average reading achievement and its standard error. 
The last two columns report the standard deviation of the achievement scores and its standard error. 
From the first two lines of results, 49.26 percent of students in Australia were girls, and 50.74 
percent were boys. The average reading achievement was 535.79 for girls (standard error of 2.67) 
and 519.20 for boys (standard error of 2.73). 

Student-level Regression Analysis 

This example is an extension of the previous example (Student-level Analysis with Achievement by 
Gender), where we will examine the difference in reading achievement between girls and boys using 
regression and determine if it is statistically significant. The results of this example also are 
presented in Exhibit 1.5 of PIRLS 2011 International Results in Reading and shown in Exhibit 2.6, in 
the column labeled “Difference.” 

For this example, the values of the variable ITSEX are recoded into variable REGSEX by 
running the special SPSS syntax file SYNTAXASGALLR3.SPS shown in Exhibit 2.9 and provided 
in the International Database. 

Exhibit 2.9: Example SPSS Program to Recode Variables for Student-level Regression Analysis 


* Compute new variable REGSEX from ITSEX . 

get file = " <datpath>\ASGALLR3 . sav" . 

compute REGSEX = ITSEX - 1 . 

value labels 

REGSEX 0 "GIRL" 

1 "BOY" . 

variable labels 

REGSEX "Recoded ITSEX (Girls = 0; Boys = 1) 
save outfile = " <datpath>\ASGALLR3 . sav" . 


The parameter <datpath> in the SPSS syntax shown in Exhibit 2.9 needs to be edited to specify 
the location of the input and output data files. 

REGSEX has a value of zero for girls and one for boys. By using REGSEX, the regression 
intercept (or constant) will be the estimated average reading achievement of girls, whereas the 
regression slope (called REGSEX in the SPSS output) will be the estimated increase in the average 
reading achievement of boys. 3 This example of a regression analysis is performed by the analysis 
module of the IEA IDB Analyzer using the following steps: 

1 . Open the Analysis Module of the IEA IDB Analyzer. 


3 This form of variable recoding, known as “dummy coding,” makes the interpretation of regression coefficients easier. It essentially transforms a 
regression analysis into an analysis of variance to test for differences among groups. 
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2. Specify the data file ASGALLR3 as the Analysis File by clicking the Select button, after having 
run the SPSS syntax file SYNTAXASGALLR3.SPS to create the variable REGSEX. 

3. Select PIRLS (Using Student Weights) as the Analysis Type. 

4. Select Regression as the Statistic Type. 

5. Select Use PVs as the Plausible Value Option. 

6. The variable IDCNTRY is selected automatically as Grouping Variables. No additional 
grouping variables are needed for this analysis. 

7. Click the Non-Plausible Values field in the Independent Variables section to activate it and 
select the variable REGSEX as the analysis variable. This is done by selecting REGSEX from the 
list of available variables and moving it to the Non-Plausible Values field by clicking the right 
arrow ( ► ) button in this section. 

8. Click the Plausible Values button in the Dependent Variable section and click the Plausible 
Values field to activate it. Select ASRREA01-05 as the analysis variable from the list of 
available variables and move it to the Plausible Values field by clicking the right arrow ( ► ) 
button in this section. 

9. The Weight Variable is selected automatically by the software; TOTWGT is selected by default 
because this example analysis uses student background data. 

10. Specify the name and the folder of the output files in the Output Files field by clicking the 
Define/Modify button. 

11. Click the Start SPSS button to create the SPSS syntax file and open it in an SPSS syntax 
window. The syntax file will be executed by opening the Run menu of SPSS and selecting the 
All menu option. If necessary, the IEA IDB Analyzer will display a prompt to confirm the 
overwriting of existing files. 

Exhibit 2.10 shows the completed analysis module for this example analysis, and Exhibit 2.1 1 
displays the results. 

Each country’s results are displayed on two lines: the first for the intercept (CONSTANT), and 
the second for the REGSEX coefficient. Generally, there will be as many lines per country as there 
are regression coefficients including the intercept. The first line of results in Exhibit 2.11, labeled 
“(CONSTANT),” is the estimated average reading achievement of girls in Australia, which was 
535.79, with a standard error of 2.67. This estimate concurs with the results obtained in the previous 
example (Exhibit 2.8). The boys in Australia had a negative average achievement difference 
of -16.58, with an estimated standard error of 3.1 1. Note that this achievement difference is 
statistically significant at the 95% confidence level. Adding the two regression coefficients together 
(535.79 - 16.58) yields the estimated average achievement of boys in Australia, 519.21, the same 
estimate (within rounding) from the previous example. As a note of interest, the IEA IDB Analyzer 
also computes standardized regression coefficients, whereby the independent variables are 
standardized to have a mean of zero and standard deviation of one (1). 
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Exhibit 2.1 0: IDB Analyzer Setup for Example Student-level Regression Analysis 


E IEA IDB Analyzer Analysis Module - (Version 3.0,47) 


O Analysis File: | C:\PIRLS2011\Data\ASGALLR3.sav [ Select | 


a 





Analysis Type: 

Statistic Type: 

Plausible Value Option: Benchmark Option: 

Missing Data Option: Number of Decimals: 


| PIRLS (Using Student Weights) 

| Regression 

[Use PVs 

| Listwise T | 1 2 » | 








0 Select Variables: 




O Output Files: C:\PIRLS2011\Data\REAGenderDiff’ 


| Modify | 
O I Start SPSS | 


Return to Main Menu 


Help | 


PIRLS (Using Student Weights) Regression Use PVs 


Exhibit 2.1 1 : Output for Example Student-Level Regression Analysis 


Regression Coefficients 
IDCNTRY 

Variable 

Regression 

Coefficient 

Regression 
Coefficient 
(s.e. ) 

Regression 

Coefficient 

(t-value) 

Stndrdzd . 
Coefficient 

Stndrdzd . 
Coefficient 
(s.e. ) 

Stndrdzd . 
Coefficient 
(t-value) 

Australia 

(CONSTANT) 

535 . 79 

2 . 67 

200.57 





REGSEX 

-16.58 

3.11 

-5.33 

- .10 

.02 

-5.42 

Austria 

(CONSTANT) 

532 . 76 

2 . 18 

244.47 





REGSEX 

-7.58 

2.31 

-3.28 

- . 06 

. 02 

-3.28 

Azerbaijan, Republic of 

(CONSTANT) 

469.57 

3.56 

131 . 76 





REGSEX 

-13 . 75 

2.34 

-5 . 87 

- . 10 

. 02 

-5 . 81 

Belgium (French) 

(CONSTANT) 

508 . 85 

3 . 11 

163 . 70 





REGSEX 

-5.34 

2.34 

-2.28 

- . 04 

. 02 

-2.27 
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Percentages of Students Reaching the PIRLS International Benchmarks 

This section describes how to use the IEA IDB Analyzer to perform analyses of student achievement 
in relation to the PIRLS International Benchmarks. As an example, we will compute the percentages 
of PIRLS 2011 students reaching the four PIRLS International Benchmarks of reading 
achievement — Advanced, High, Intermediate, and Low — using the merged ASGALLR3 data file. 
These results, presented in Exhibit 2.2 of PIRLS 2011 International Results in Reading, are shown 
here in Exhibit 2.12. This example is performed by the analysis module of the IEA IDB Analyzer 
using the following steps: 

1 . Open the Analysis Module of the IEA IDB Analyzer. 

2. Specify the data file ASGALLR3 as the Analysis File by clicking the Select button. 

3. Select PIRLS (Using Student Weights) as the Analysis Type. 

4. Select Benchmarks as the Statistic Type. 

5. Select the Cumulative option under the Benchmark Option drop-down menu to get cumulated 
percentages of students reaching the PIRLS international benchmarks. 

6. The variable IDCNTRY is selected automatically as Grouping Variables. No additional 
grouping variables are needed for this analysis. 

7. Specify the achievement scores to be used for the analysis by clicking the Plausible Values field. 
Select ASRREA01-05 from the list of available variables and move it to the Plausible Values 
field by clicking the right arrow ( ► ) button in this section. 

8. Specify the PIRLS International Benchmarks — 400, 475, 550, and 625 (Low, Intermediate, High, 
and Advanced, respectively). Enter these four values in the Achievement Benchmarks field, 
each separated by a blank space. 

9. The Weight Variable is selected automatically by the software; TOTWGT is selected by default 
because this example analysis uses student background data. 

10. Specify the name and the folder of the output files in the Output Files field by clicking the 
Define/Modify button. 

11. Click the Start SPSS button to create the SPSS syntax file and open it in an SPSS syntax 
window. The syntax file will be executed by opening the Run menu of SPSS and selecting the 
All menu option. If necessary, the IEA IDB Analyzer will display a prompt to confirm the 
overwriting of existing files. 
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Exhibit 2.1 2: Example Exhibit of International Benchmark Analysis, Taken from 
PIRLS 2011 International Results in Reading (Exhibit 2.2) 


Exhibit 2.2: Performance at the International Benchmarks of 
Reading Achievement 


PIRLS 2011I 


Country 

Percentages of Students Reaching 
International Benchmarks 

• Advanced 
OHigh 

• Intermediate 
O Low 

Advanced 

International 

Benchmark 

(625) 

High 

International 

Benchmark 

(550) 

Intermediate 

International 

Benchmark 

(475) 


2 Singapore 


2 Denmark 
Hungary 
Bulgaria 


2 Croatia 


Australia 

Italy 


Germany 

Portugal 
Sweden 
Czech Republic 
Slovak Republic 


Slovenia 


Poland 


Romania 


t Netherlands 


1 2 Lithuania 


France 


Benchmark 

(400) 


Russian Federation 


t Northern Ireland 

Finland 

1 England 

3 Hong Kong SAR 

2 United States 
Ireland 

3 Israel 

New Zealand 
2 Canada 
Chinese Taipei 


-O— 

— O- 




-o — 
— o— 



-• — 


Trinidad and Tobago 
United Arab Emirates 

1 Georgia 

2 1 Belgium (French) 

2 Qatar 

* Norway 

Iran, Islamic Rep. of 
Colombia 
Saudi Arabia 
2 Azerbaijan 
V Oman 


Indonesia 


• O 


* Morocco CD- 

International Median 


24(1.6) 


62(1.8) 


87 (1.1) 


97 (0.4) 


19(1.2) 


63 (1.7) 


92 (1.1) 


99 (0.2) 


19(1.2) 


58 (1.4) 


87 (0.9) 


97 (0.6) 


18 (0.9) 


63 (1.3) 


92 (0.7) 


99 (0.2) 


18(1.1) 


54(1.3) 


83 (1.1) 


95 (0.5) 


18(1.2) 

■■ 

67 (1.5) 

■I 

93 (0.8) 

■1 

99 (0.2) 


17(0.7) 


56 (0.8) 


86 (0.6) 


98 (0.3) 


16(0.9) 

■■ 

53 (1.4) 

■I 

85 (0.8) 


97 (0.5) 


15 (0.9) 


49(1.3) 


80(1.3) 


93 (0.8) 


14(0.7) 


45 (1.1) 


75 (0.9) 


92 (0.5) 


13 (0.7) 


51 (1.1) 


86 (0.6) 


98 (0.2) 


13 (0.9) 

■■ 

55 (1.3) 

II 

87 (0.7) 

■1 

98 (0.3) 


■ 12 (0.8) 


55 (1.2) 


88 (0.8) 


99 (0.2) 


12 (0.9) 

■■ 

48(1.5) 

II 

81 (1.2) 

■1 

95 (0.7) 


11 (0.8) 


45 (2.0) 


77 (1.9) 


93 (1.0) 


11 (0.7) 

II 

54 (1.3) 

II 

90 (0.7) 

■1 

99 (0.2) 


10 (0.7) 


42 (1.1) 


76 (1.0) 


93 (0.7) 


10(0.7) 


46(1.4) 


85 (1.1) 


98 (0.4) 


10 (0.8) 


46(1.4) 


85 (1.0) 


98 (0.3) 


9(1.1) 

II 

47 (1.8) 

II 

84(1.2) 

■1 

98 (0.5) 

■ 

9 (0.8) 


47 (1.6) 


85 (1.0) 


98 (0.3) 


8(0.9) 

II 

50 (1.4) 

II 

87 (0.9) 

■1 

98 (0.5) 


8 (0.6) 


44(1.5) 


82 (1.3) 


96 (0.8) 


8(0.7) 


42 (1.2) 


79 (0.9) 


95 (0.6) 


7(0.6) 


39 (1.2) 


77 (0.9) 


95 (0.5) 


7(0.7) 

■■ 

32 (1.6) 

■I 

65 (2.1) 

■1 

86 (1.5) 


> 7(0.5) 


48(1.5) 


90 (0.8) 


100 (0.2) 


6(0.5) 

■■ 

39 (1.4) 

II 

80(1.2) 

■1 

97 (0.4) 


5(0.5) 


35 (1.6) 


75 (15) 


95 (0.8) 


5(0.5) 


39 (1.5) 


80 (0.9) 


97 (0.3) 


4 (0.4) 


24 (0.7) 


55 (0.8) 


78 (0.6) 


4(0.5) 

II 

31 (1.3) 

II 

72 (U) 

■1 

94 (0.7) 


3(0.5) 


19 (1.4) 


50 (1.9) 


78 (1.5) 


3 (0.3) 

II 

14 (0.6) 

II 

38 (1.0) 

■1 

64 (0.9) 


2 (0.3) 


21 (1.2) 


60(1.6) 


86(1.4) 


2 (0.5) 

II 

25 (1.4) 

II 

70 (1.7) 

■1 

94 (1.1) 


2(0.5) 


12 (1.2) 


34 (1.4) 


60 (1.5) 


2(0.4) 


25 (1.5) 


71 (1.3) 


95 (0.7) 


1 (0.2) 


13 (0.9) 


45 (1.6) 


76 (1.1) 


1 (0.3) 

II 

10 (1.3) 


38(2.1) 

■1 

72 (1.9) 


1 (0.2) 


8 0.0) 


34 (2.0) 


65 (1.9) 


0(0.3) 

II 

9 (0.9) 

II 

45 (2.1) 

■1 

82 (1.6) 


0(0.1) 


5 (0.4) 


21 (0.9) 


47 (1.2) 


0(0.1) 

II 

4 (0.6) 

II 

28 (1.9) 

■1 

66 (2.2) 


0 (0.0) 


1 (0.2) 


7(0.7) 


21 (1.3) 





• Advanced 


Advanced 

High 

Intermediate 

Low 

Country 

Percentages of Students Reaching 

OHigh 


International 

International 

International 

International 

International Benchmarks 

• Intermediate 


Benchmark 

Benchmark 

Benchmark 

Benchmark 



O Low 


(625) 

(550) 

(475) 

(400) 


Sixth Grade Participants 

1 * Kuwait • O • O 

Botswana # O # O 

Honduras • O # O 

Morocco m Q— — ■ # ■ # 


Benchmarking Participants 0 

1 3 Florida, US 
2 Ontario, Canada 
2 Alberta, Canada 
Quebec, Canada 
Dubai, UAE 
Andalusia, Spain 
v Eng/Afr (5) - RSA 
Abu Dhabi, UAE 
Maltese - Malta 





2 (0.4) 

11 (1.0) 

34 (1.6) 

58(2.2) 

1 (0.4) 

9 0.3) 

27 (1.8) 

56 (1.8) 

1 (0.4) 

10 (1.4) 

38 (2.2) 

74 (2.3) 

1 (0.1) 

7 (0.6) 

30 (1.6) 

61 (1.9) 


22(1.7) 

61 (1.7) 

91 (1.1) 

98 (0.4) 

15 0.3) 

54 0.7) 

85 (1.1) 

97 (0.4) 

13(1.0) 

51 (1.6) 

85 (U) 

97 (0.5) 

7(0.7) 

43 (1.9) 

85 (1.0) 

98 (0.3) 

6 (0.4) 

26 (0.9) 

54 (1.0) 

75 (0.8) 

4(0.4) 

31 (1.5) 

73 (1.3) 

95 (0.7) 

4(0.8) 

14 (1.5) 

34 (2.6) 

57 (2.8) 

2 (0.6) 

10 (1.2) 

32 (1.9) 

60 (1.9) 

1 (0.2) 

14 (0.7) 

45 (0.9) 

74 (0.9) 


100 


M Average achievement not reliably measured because the percentage of students with achievement too low for estimation exceeds 25%. 

Y Reservations about reliability of average achievement because the percentage of students with achievement too low for estimation does not exceed 25% but exceeds 1 5%. 
See Appendix C.2 for target population coverage notes 1, 2, and 3. See Appendix C.5 for sampling guidelines and sampling participation notes t, X, and i. 

( ) Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 
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Exhibit 2.13 shows the completed analysis module for this example analysis, and Exhibit 2.14 
presents the results. 

Exhibit 2.1 3: IDB Analyzer Setup for Example International Benchmark Analysis 


r IEA IDB Analyzer: Analysis Module - (Version 3.0.47) 


Q Analysis File: | C:\PIRLS2011\Data\ASGALLR3.sav | Select 


f 

0 

Analysis Type: 

Statistic Type: 

Plausible Value Option: Benchmark Option: 

Missing Data Option: Number of Decimals: 


f PIRLS (Using Student Weights) 

w \ | Benchmarks 

Use PVs | Cumulative 

H 12 H 


V 






0 Select Variables: 




O Output Files: C:\PIRLS2011\Data\REAIBM* 


| Modify | 
0 | Start SPSS | 


Return to Main Menu 


][ 


Help | 


PIRLS (Using Student Weights) Benchmarks Use PVs Cumulative 


Exhibit 2.1 4: Output for Example International Benchmark Analysis 


Percent reaching benchmarks 

(400 475 550 625] 

l of ASRREA0 





Performance 


N of 

Sum of 


Percent 

* COUNTRY ID* 

Group 



Cases 

TOTWGT 

Percent 

(s .e. ) 

Australia 

At 

or 

Above 

400 

5664 

233459 

92.93 

.67 


At 

or 

Above 

475 

4591 

189968 

75.62 

1.03 


At 

or 

Above 

550 

2514 

105274 

41 . 91 

1 . 14 


At 

or 

Above 

625 

591 

24945 

9.93 

.65 

Austria 

At 

or 

Above 

400 

4519 

76065 

97.10 

.35 


At 

or 

Above 

475 

3710 

62965 

80.38 

.94 


At 

or 

Above 

550 

1795 

30590 

39.05 

1.50 


At 

or 

Above 

625 

261 

4088 

5.22 

.54 

Azerbaijan, Republic of 

At 

or 

Above 

400 

4014 

85145 

81.86 

1.60 


At 

or 

Above 

475 

2212 

46977 

45 . 16 

2 . 10 


At 

or 

Above 

550 

389 

9302 

8 . 94 

. 93 


At 

or 

Above 

625 

11 

459 

.44 

.28 

Belgium (French) 

At 

or 

Above 

400 

3527 

46784 

93 . 79 

1 . 08 


As 

or 

Above 

475 

2664 

35110 

70.39 

1.67 


At 

or 

Above 

550 

971 

12719 

25.50 

1.39 


At 

or 

Above 

625 

91 

1122 

2.25 

.49 
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As shown in the first few lines of results, 92.93 percent of the students in Australia performed at 
or above the Low International Benchmark of 400, with a standard error of 0.67; 75.62 percent of the 
students reached the Intermediate International Benchmark, with a standard error of 1.03; 

41.91 percent of the students reached the High International Benchmark, with a standard error 
of 1.14; and 9.93 percent of the students reached the Advanced International Benchmark, with a 
standard error of 0.65. 

Computing Correlations with Background Variables and Achievement Scores 

In addition to the analyses described above, the IEA IDB Analyzer (version 3.0) is able to compute 
correlations among background variables as well as between background variables and achievement 
scores. While these types of analyses will not be demonstrated here, the steps for conducting them 
are the same as those described previously: select the grouping variables, the analysis variables, the 
achievement scores (if necessary), and confirm the weight variable. The output will display, for each 
group defined by the grouping variables, the correlation coefficients for each possible pair of 
variables. When using only background variables, the diagonal and elements above the diagonal of 
the correlation matrix are displayed with their corresponding standard errors. When using 
achievement scores, a single column is displayed containing the correlations between each of the 
background variables specified in the model and the achievement scores selected. 

Calculating Percentiles of Achievement 

The Percentiles statistic type is an additional tool provided by the IEA IDB Analyzer (version 3.0) 
for analyzing the PIRLS 2011 data. This procedure will compute the percentiles of a student 
achievement distribution for a specified set of plausible values and within any specified subgroups of 
students, along with appropriate standard errors. 

2.6 PIRLS/prePIRLS Analyses with Home Background Data 

Analyzing home background data from the PIRLS 201 1 International Database requires that the 
home background data files (ASH) be merged with the student background data files (ASG) in order 
to retrieve the achievement scores and the required sample design variables. To illustrate how to 
perform analyses based on home background data, we will use an example that computes the average 
scale score for one of the newly developed contextual scales, along with the percentages of students, 
with their average reading achievement, for the categories of the scale’s corresponding index. 

Analyses with a Contextual Scale 

PIRLS 2011 has innovated in its reporting of contextual data by creating contextual scales based on 
Rasch modeling. 4 A number of contextual scales were reported in PIRLS 2011 International Results 
in Reading and are available in the International Database for analysis. Each contextual scale 
variable is a Rasch score with an international mean of 10 and an international standard deviation 
of 2. An index was derived from each scale that divides the range of scores into usually three 
categories: the most desirable scores (high values), the least desirable scores (low values), and the 
remaining scores in between. 


4 The contextual scales are described in the context questionnaire scales section of Methods and Procedures in TIMSS and PIRLS 2011. 
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These contextual scales and their corresponding indices were reported in PIRLS 2011 
International Results in Reading. Exhibit 2.15 shows one such example, Exhibit 4.6 from PIRLS 
2011 International Results in Reading, analyzing students’ Early Literacy Activities Before 
Beginning Primary School as reported by their parents. Results on the Rasch scale are reported for 
each country as an “Average Scale Score” and its corresponding index is reported as the percentages 
of students in each category — Often, Sometimes, and Never or Almost Never — along with the 
average achievement in each category. The Rasch scores for this contextual scale are contained in the 
ASBHELA variable; the corresponding index is found in the ASDHELA variable. 

This example will replicate both the average scale score of the Early Literacy Activities 
contextual scale and the percentages of students, with their average reading achievement, in each 
category of its index. This will be done in two steps, both using the merged ASHALLR3 data file 
that combines the home background data files and the student background data files. 

The first step will compute the average scale score using the contextual scale variable 
ASBHELA. It is performed by the analysis module of the IEA IDB Analyzer using the following 
steps: 

1 . Open the Analysis Module of the IEA IDB Analyzer. 

2. Specify the data file ASHALLR3 as the Analysis File. 

3. Select PIRLS (Using Student Weights) as the Analysis Type. 

4. Select Percentages and Means as the Statistic Type. 

5. Select None Used as the Plausible Value Option, because we will not use any achievement 
scores for this part of the analysis. 

6. Specify the variable ASBHELA as the analysis variable and activate it by clicking the Analysis 
Variables field. Select ASBHELA from the list of available variables and move it to the 
Analysis Variables field by clicking the right arrow ( ► ) button in this section. 

7. The Weight Variable is selected automatically by the software; TOTWGT is selected by default 
because this example analysis uses home background data linked to student background data. 

8. Specify the name and the folder of the output files in the Output Files field by clicking the 
Define/Modify button. 

9. Click the Start SPSS button to create the SPSS syntax file and open it in an SPSS syntax 
window. The syntax file can be executed by opening the Run menu of SPSS and selecting the 
All menu option. If necessary, the IEA IDB Analyzer will display a prompt to confirm the 
overwriting of existing files. 
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Exhibit 2.1 5: Example Exhibit of a Contextual Scale, Taken from 

PIRLS 2011 International Results in Reading (Exhibit 4.6) 


Exhibit 4.6: Early Literacy Activities Before Beginning Primary School 


PIRLS 20111 


Reported by Parents 

Students were scored according to their parents' frequency of doing the nine activities on the Early Literacy Activities scale. Students Often engaged 
in early literacy activities had a score on the scale of at least 10.7, which corresponds to their parents "often" doing five of the nine activities with them 
and "sometimes" doing the other four, on average. Students Never or Almost Never engaged in such activities had a score no higher than 6.2, which 
corresponds to parents "never or almost never" doing five of the nine activities with them and "sometimes" doing the other four, on average. All other 
students had parents who Sometimes engaged them in early literacy activities. 


Often 


Sometimes 


Never or Almost Never 


Country 


Percent 


Average 


Percent 


Average 


Percent 


Average 


Scale Score 




of Students 


Achievement 


of Students 


Achievement 


of Students 


Achievement 




Russian Federation 


61 (1.3) 


576 (2.7) 


38 (1.2) 


558 (3.4) 


1 (0.3) 




11.1 (0.06) 


Northern Ireland 

s 

59(1.3) 

■1 

582 (3.5) 

■■ 

41 (1.4) 

■1 

559 (3.7) 

II 

0(0.2) 

■1 

— 

1 I 

11.2 (0.06) 

■ 

New Zealand 

s 

55 (1.0) 


567 (2.7) 


44 (1.0) 


529 (2.5) 


1 (0.1) 


~~ 


11.0 (0.05) 


Australia 

s 

52 (1.4) 

■1 

555 (3.0) 

■■ 

46 (1.3) 

■1 

528 (3.4) 

II 

1 (0.3) 

■1 


1 1 

10.8 (0.06) 

■ 

Georgia 


52 (1.4) 


498 (2.6) 


47(1.3) 


479 (4.0) 


1 (0.2) 


~~ 


10.7 (0.06) 


Canada 

r 

51 (0.9) 

■1 

566 (1.9) 

■I 

48 (0.9) 

■1 

541 (1.8) 

II 

1 (0.1) 

■1 


1 1 

10.7 (0.04) 

■ 

Ireland 


50 (0.9) 


569 (2.3) 


49 (0.8) 


542 (2.6) 


1 (0.1) 


~~ 


10.8 (0.04) 


Croatia 

■■ 

50 (0.9) 

■1 

562 (2.2) 

■■ 

49 (0.9) 

■1 

544 (1.9) 

II 

0(0.1) 

■1 

— 

1 | 

10.7 (0.03) 

■ 

Slovenia 


48(1.2) 


543 (2.3) 


51 (1.2) 


522 (2.6) 


0(0.1) 




10.6 (0.04) 


Israel 

r 

48 (1.0) 

■1 

563 (3.0) 

■■ 

51 (1.0) 

■1 

534 (3.5) 

II 

1 (0.2) 

■1 

— 

1 | 

10.6 (0.04) 

■ 

Italy 


48 (0.9) 


553 (2.4) 


51 (1.0) 


537 (2.6) 


1 (0.2) 


~~ 


10.5 (0.03) 


Slovak Republic 

■■ 

47 (0.9) 

■1 

547 (2.9) 

■■ 

51 (0.9) 

■1 

530 (2.5) 

II 

2(0.6) 

■1 

— 

1 I 

10.5 (0.05) 

■ 

Trinidad and Tobago 


47 (1.1) 


497 (4.0) 


52(1.1) 


456 (4.1) 


1 (0.3) 


~~ 


10.5 (0.05) 


Malta 

■■ 

45 (0.9) 

■1 

507 (1.9) 

■■ 

54 (0.9) 

■1 

463 (2.7) 

II 

1 (0.2) 

■1 

— 

1 1 

10.4 (0.04) 

■ 

Spain 


44 (1.0) 


528 (2.7) 


55 (1.0) 


507 (2.7) 


1 (0.2) 


~~ 


10.4 (0.03) 


Poland 

■■ 

43 (0.8) 

■1 

544 (2.8) 

■■ 

56 (0.8) 

■1 

514 (2.1) 

II 

1 (0.3) 

■1 

— 

1 1 

10.4 (0.03) 

■ 

Hungary 


43 (0.8) 


553 (2.8) 


56 (0.8) 


535 (3.2) 


1 (0.4) 


~~ 


10.3 (0.04) 


Czech Republic 

■■ 

40 (1.0) 

■1 

555 (2.6) 

■■ 

60(1.0) 

■1 

542 (2.3) 

II 

1 (0.2) 

■1 


1 1 

10.3 (0.03) 

■ 

Netherlands 

s 

40 (0.8) 


559 (3.1) 


60 (0.8) 


551 (2.0) 


1 (0.2) 




10.2 (0.03) 


Bulgaria 


39 (1.4) 

■1 

559 (3.1) 

■I 

51 (1.0) 

■1 

529 (3.7) 

II 

9 (1.4) 

■1 

455 (15.3) 

1 1 

9.7 (0.12) 

■ 

Romania 


38 (1.5) 


529 (4.1) 


54(1.3) 


494 (4.5) 


8 (1.0) 


423 (8.9) 


9.9 (0.09) 


Germany 

r 

38 (0.9) 

■1 

555 (2.8) 

■■ 

61 (0.9) 

■1 

543 (2.2) 

II 

1 (0.2) 

■1 


1 1 

10.2 (0.03) 

■ 

Norway 


37 (1.4) 


524 (2.5) 


63 (1.4) 


500 (2.2) 


1 (0.2) 




10.0 (0.06) 


France 

■■ 

36 (0.7) 

■1 

536 (2.6) 

■I 

63 (0.7) 

■1 

515 (2.7) 

II 

1 (0.2) 

■1 


1 1 

10.0 (0.03) 

■ 

Lithuania 


36 (0.9) 


541 (1.9) 


63 (0.9) 


524 (2.5) 


2 (0.2) 




10.0 (0.03) 


Austria 

■■ 

35 (1.0) 

■1 

543 (2.1) 

■■ 

63 (1.1) 

■1 

523 (2.5) 

II 

1 (0.2) 

■1 

— 

I | 

10.0 (0.03) 

■ 

Portugal 


35 (1.1) 


558 (2.8) 


63 (1.1) 


535 (2.6) 


2 (0.4) 




10.0 (0.05) 


Sweden 

■■ 

34 (1.0) 

■1 

562 (2.9) 

■■ 

64(1.0) 

■1 

537 (2.2) 

II 

2(0.2) 

■1 


1 I 

9.9 (0.04) 

■ 

Colombia 


34(1.1) 


457 (5.7) 


63 (1.0) 


448 (3.8) 


3 (0.4) 


409(11.0) 


9.9 (0.06) 


Denmark 

■■ 

32 (0.9) 

■1 

567 (2.2) 

■■ 

67 (0.9) 

■1 

550 (1.9) 

II 

1 (0.2) 

■1 

— 

1 I 

9.9 (0.03) 

■ 

Belgium (French) 


30 (0.8) 


524 (2.9) 


67 (0.7) 


501 (3.2) 


3 (0.4) 


482 (10.3) 


9.7 (0.04) 


Qatar 

■■ 

28 (1.0) 

■1 

458 (6.0) 

■■ 

69(1.0) 

■1 

420 (3.1) 

II 

3 (0.3) 

■1 

390 (11.7) 

1 I 

9.7 (0.05) 

■ 

Finland 


27 (0.9) 


583 (2.9) 


72 (0.9) 


564 (1.9) 


1 (0.1) 


~~ 


9.7 (0.03) 


United Arab Emirates 

■■ 

27(0.5) 

■■ 

480 (2.8) 

■■ 

71 (0.5) 

■1 

430 (2.3) 

II 

3 (0.2) 

■1 

392 (7.4) 

1 I 

9.6 (0.03) 

■ 

Saudi Arabia 


26(1.3) 


455 (5.2) 


70(1.4) 


426 (4.2) 


4(0.8) 


360 (16.2) 


9.5 (0.06) 


Singapore 

■■ 

26 (0.7) 

■1 

595 (3.6) 

■■ 

69 (0.7) 

■1 

561 (3.3) 

II 

5 (0.3) 

■1 

543 (6.2) 

1 I 

9.4 (0.03) 

■ 

Azerbaijan 


23 (1.3) 


467 (4.8) 


72(1.3) 


463 (3.5) 


4(0.7) 


439 (7.6) 


9.5 (0.07) 


Indonesia 

■■ 

23 (1.3) 

■1 

445 (5.8) 

■■ 

72(1.3) 

■1 

427 (4.1) 

II 

4(0.5) 

■1 

409 (6.5) 

1 I 

9.4 (0.07) 

■ 

Oman 


19 (0.4) 


429 (3.6) 


76 (0.5) 


385 (3.2) 


5 (0.4) 


354 (7.2) 


9.2 (0.03) 


Morocco 

■■ 

17 (0.9) 

■1 

321 (5.6) 

■■ 

64(1.5) 

■1 

314 (4.5) 

II 

19 (1.9) 

■1 

302 (13.6) 

1 I 

8.4 (0.13) 

■ 

Iran, Islamic Rep. of 


15 (0.6) 


474 (3.6) 


77 (0.8) 


460 (3.0) 


8 (0.8) 


411 (8.0) 


8.9 (0.05) 


Chinese Taipei 

■■ 

14 (0.6) 

■1 

577 (3.5) 

■■ 

76 (0.8) 

■1 

553 (1.8) 

II 

10 (0.6) 

■1 

526 (4.8) 

1 I 

8.7 (0.04) 

■ 

Hong Kong SAR 


12 (0.6) 


588 (3.7) 


80 (0.6) 


571 (2.3) 


8 (0.5) 


560 (3.7) 


8.7 (0.03) 


International Avg. 


37 (0.2) 


529 (05) 


60 (0.2) 


506 (0.5) 


3 (0.1) 


430 (2.6) 

1 



Sixth Grade Participants 
















Honduras 


30 (1.5) 


461 (8.3) 


63 (1.3) 


447 (4.2) 


7(0.7) 


426 (12.2) 


9.6 (0.08) 


Kuwait 

s 

23 (1.2) 

■1 

448 (9.3) 

■I 

73 (1.2) 

■1 

420 (6.5) 

■I 

4(0.5) 

■1 

378 (14.2) 

■ I 

9.4 (0.06) 

■ 

Morocco 


16 (0.7) 


437 (6.7) 


67 (1.2) 


432 (4.2) 


16(1.3) 


406 (7.5) 


8.5 (0.08) 


Botswana 

■■ 

15(1.1) 

■l 

464 (8.7) 

■■ 

74(1.2) 

■l 

420 (3.9) 

■■ 

11(1.0) 

■l 

388 (6.2) 

■ ■ 

8.6 (0.08) 

■ 

Benchmarking Participants 0 

Ontario, Canada 

r 

54(1.3) 


566 (3.1) 


45 (1.3) 


545 (3.2) 


1 (0.2) 




10.9 (0.05) 


Alberta, Canada 

r 

52(1.3) 

■1 

568 (3.6) 

■■ 

47 (1.3) 

■1 

542 (3.7) 

II 

1 (0.2) 

■1 

— 

1 I 

10.8 (0.05) 

■ 

Maltese - Malta 


45 (0.8) 


481 (2.0) 


54 (0.8) 


447 (1.9) 


1 (0.2) 




10.4 (0.03) 


Andalusia, Spain 

■■ 

43 (0.9) 

■1 

530 (2.5) 

■■ 

56 (0.9) 

■1 

508 (2.7) 

II 

1 (0.2) 

■1 

— 

■ I 

10.3 (0.04) 

■ 

Quebec, Canada 


40 (1.0) 


554 (3.0) 


59 (1.0) 


531 (2.4) 


2 (0.3) 




10.2 (0.04) 


Eng/Afr (5) - RSA 

r 

34 (1.5) 

■1 

465 (8.5) 

■■ 

61 (1.4) 

■1 

405 (7.2) 

II 

4(0.7) 

■1 

358 (14.5) 

1 I 

9.8 (0.09) 

■ 

Dubai, UAE 


34 (0.8) 


521 (2.4) 


64 (0.8) 


463 (2.6) 


2(0.2) 




9.9 (0.03) 


Abu Dhabi, UAE 

■■ 

24(1.1) 

■1 

466 (6.4) 

■■ 

73 (1.0) 

■1 

416 (4.4) 

■I 

3 (0.4) 

■1 

379 (9.6) 

1 I 

9.5 (0.05) 

■ 


0 Republic of South Africa (RSA) tested 5th grade students receiving instruction in English (ENG) or Afrikaans (AFR). 

England and the United States did not administer the Home Questionnaire. 

Centerpolnt of scale set at 10. 

( ) Standard errors appear in parentheses. Because of rounding some results may appear Inconsistent 
A tilde (~) Indicates Insufficient data to report achievement 

An "x" indicates data are available for at least 70% but less than 85% of the students. An “s' Indicates data are available for at least 50% but less than 70% of 
the students. 
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Exhibit 2.16 shows the completed analysis module for this example analysis, and Exhibit 2.17 
displays the results. 

Exhibit 2.16: IDB Analyzer Setup for Example Home Background Analysis with a Contextual Scale (Step 1 ) 


f IEA IDB Analyzer Analysis Module - (Version 3 . 0 . 47 ) 


O Analysis File: | C:\PIRLS2011\Data\ASHALLR3.sav | Select 


f 

0 

Analysis Type: 

Statistic Type: 

Plausible Value Option: Benchmark Option: 

Missing Data Option: Number of Decimals: 


f PIRLS (Using Student Weights) 

* | | Percentages and Means » 

( None Used ' r 

' |2 -1 


V 






0 Select Variables: 



I Name 

Description 

H 1 11 

40ASBH17A 

GEN\ADD\LVL OF EDUCATION\FATHER * 

40ASBH17B 

GEN\ADD\LVL OF EDUCATtON\MOTHER 

4$ASBH18 

GEN\ADD\LVL OF EDUCA7TON\CHILD 

4$ASBH19A 

GEN\ADD\EM PLOYM ENT SITU ATIO N\FATH E R 

4$ASBH19B 

GEN\ADD\EM PLOYM ENT SITU ATIO N\MOTH E R 

40ASBH2OA 

GEN\ADD\WHAT KIND OF MAIN JOBXFATHER 

4$ASBH20B 

GEN\ADD\WHAT KIND OF MAIN JOBXMOTHER 

SSYSTEM 

‘SYSTEM ID HOME RLE* 

AS DAG E 

‘STUDENTS AGE* 

<j$ASBGHRL 

‘HOME RESOURCES FOR LEARNING/SC L‘ 

4$ASDGHRL 

‘HOME RESOURCES FOR LEARNING/IDX* 

^ASBHPLR 

‘PARENTS LIKE READING/SCL‘ 

4$asdhplr 

‘PARENTS LIKE READING/IDX* 

40ASDHELA 

‘EARLY LITERACY ACTIVmES BEFORE BEGINNING PRIMARY... 

<J$ASBHELT 

‘EARLY LITERACY TASKS/SCL* 

40ASDHELT 

‘EARLY LITERACY TASKS/IDX* 

^ASBHENA 

‘EARLY NUMERACY ACTIVITIES BEFORE BEGINNING PRIMAR... 

40ASDHENA 

‘EARLY NUMERACY ACTIVITIES BEFORE BEGINNING PRIMAR... 

^ASBHENT 

‘EARLY NUMERACY TASKS/SCL* 

^ASDHENT 

‘EARLY NUMERACY TASKS/IDX* T 


O Output Files: C:\PIRLS2011\Data\ASBGELA.’ 


| Modify | 
0 [ Start SPSS | 


| Return to Main Menu 1 1 Help 


PIRLS (Using Student Weights) Percentages and Means None Used 


Exhibit 2.1 7: Output for Example Home Background Analysis with a Contextual Scale (Step 1 ) 


Average for ASBHELA by ( IDCNTRY) 


* COUNTRY ID* 

N of 
Cases 

Sum of 
TOTWGT 

Percent 

Percent 
(s .e. ) 

ASBHELA 

(Mean) 

ASBHELA 
(s .e. ) 

Std.Dev 

Std.Dev. 
(s .e. ) 

Percent 

Missing 

Australia 

3232 

135420 

. 92 

. 03 

10 . 84 

.06 

2 . 09 

.03 

46.09 

Austria 

4393 

74088 

.50 

.02 

9.98 

.03 

1 . 74 

.03 

5.42 

Azerbaijan, Republic of 

4509 

96026 

. 65 

. 03 

9.47 

. 07 

1 . 82 

. 05 

7 . 68 

Belgium (French) 

3383 

44793 

.30 

. 01 

9.69 

.04 

1 . 74 

.03 

10.20 


As shown in the first line of the results, students in Australia scored 10.84 on the Early Literacy 
Activities contextual scale, with a standard error of 0.06 Note that this is well above the international 
average of 10, the difference being statistically significant at the 95% confidence level. The IEA IDB 
Analyzer also computes the standard deviation of the contextual scale and its standard error. 
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In the second step, we will compute the percentages of students, with their average reading 
achievement, in each category of the index variable ASDHELA. This is performed by the analysis 
module of the IEA IDB Analyzer using the following steps: 

1 . Open the Analysis Module of the IEA IDB Analyzer. 

2. Specify the data fde ASHALLR3 as the Analysis File. 

3. Select PIRLS (Using Student Weights) as the Analysis Type. 

4. Select Percentages and Means as the Statistic Type. 

5. Select Use PVs as the Plausible Value Option, because we will be computing average reading 
achievement by the grouping variable ASDHELA. 

6. Specify the variable ASDHELA as a second grouping variable and activate it by clicking the 
Grouping Variables field. Select ASDHELA from the list of available variables and move it to 
the Grouping Variables field by clicking the right arrow ( ► ) button in this section. 

7. Specify the achievement scores to be used for the analysis and activate them by clicking the 
Plausible Values field. Select ASRREA01-05 from the list of available variables and move it to 
the Plausible Values field by clicking the right arrow ( ► ) button in this section. 

8. The Weight Variable is selected automatically by the software; TOTWGT is selected by default 
because this example analysis uses home background data linked to student background data. 

9. Specify the name and the folder of the output files in the Output Files field by clicking the 
Define/Modify button. 

10. Click the Start SPSS button to create the SPSS syntax file and open it in an SPSS syntax 
window. The syntax file can be executed by opening the Run menu of SPSS and selecting the 
All menu option. If necessary, the IEA IDB Analyzer will display a prompt to confirm the 
overwriting of existing files. 

Exhibit 2.18 shows the completed analysis module for this example analysis and the results are 
presented in Exhibit 2.19. 

As shown in the first three lines of the results, 52.34 percent of students in Australia often 
experienced early literacy activities before beginning primary school (standard error of 1.39), and 
their average reading achievement was 554.83 (standard error of 3.01); 46.23 percent sometimes 
experienced early literacy activities (standard error of 1.31), and their average reading achievement 
was 527.72 (standard error of 3.37); and 1.44 percent of students never or almost never experienced 
early literacy activities (standard error of 0.32), and their average reading achievement was 490.46 
(standard error of 23.75). The large standard error of this last estimated average achievement, based 
on a very small sample size and accounting for less than 2 percent of the target population, should be 
a note of caution as to the poor reliability of that estimate. 
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Exhibit 2.1 8: IDB Analyzer Setup for Example Home Background Analysis with a Contextual Scale (Step 2) 


a IEA IDB Analyzer: Analysis Module - (Version 3.0.47) 


O Analysis File: C:\PIRLS2011\Data\ASHALLR3.s< 


I Select 


0 

Analysis Type: 

Statistic Type: 

Plausible Value Option: Benchmark Option: 

Missing Data Option: Number of Decimals: 


[ PIRLS (Using Student Weights) 

w | | Percentages and Means w 

| [Use PVs - | 

|2 -1 









0 Select Variables: 



A Grouping Variables: 13 Exclude Missing From Analysis 


► \0 IDCNTRY 
4$ASDHELA 


‘COUNTRY ID* 

•EARLY LITERACY ACTIVITIES BEFORE BEGINNING PRI... 


v Analysis Variables: 
A Plausible Values: 


□H] 


X- ASRREA01-05 1ST TO 5TH PLAUSIBLE VALUE: OVERALL READING P... 


v Dependent Variable: 

* Weight Variable: 




TOTAL STUDENT WEIGHT* 


v Achievement Benchmarks / Percentiles: 


O Output Files: C:\PIRLS2011\Data\ASDHELA* 


I 


Return to Main Menu 1 1 Help 


0| Start SPSS 



PIRLS (Using Student Weights) Percentages and Means Use PVs 


Exhibit 2.1 9: Output for Example Home Background Analysis with a Contextual Scale (Step 2) 


Average for ASRREAO by 

IDCNTRY ASDHELA 










* EARLY LITERACY 
ACTIVITIES BEFORE 
BEGINNING PRIMARY 

N of 

Sum of 


Percent 

ASRREAO 

ASRREAO 


Std.Dev. 

* COUNTRY ID* 

SCHOOL/ IDX* 

Cases 

TOTWGT 

Percent 

(s.e. ) 

(Mean) 

(s.e.) 

Std.Dev 

(s.e.) 

Australia 

OFTEN 

1745 

70874 

52.34 

1 .39 

554 . 83 

3 . 01 

73 . 98 

1 . 81 


SOMETIMES 

1450 

62600 

46.23 

1.31 

527 . 72 

3.37 

79.08 

2.59 


NEVER OR ALMOST NEVER 

37 

1947 

1.44 

.32 

490.46 

23 . 75 

99.22 

13.46 

Austria 

OFTEN 

1592 

26099 

35.23 

1.05 

543.39 

2 . 10 

59.88 

1.52 


SOMETIMES 

2732 

46913 

63.32 

1 . 07 

523.49 

2.50 

63 . 66 

1.29 


NEVER OR ALMOST NEVER 

69 

1077 

1.45 

.21 

501 . 83 

7 . 79 

63.53 

7.58 

Azerbaijan, Republic of 

OFTEN 

1119 

22528 

23.46 

1.33 

466.95 

4 . 84 

67.88 

2.61 


SOMETIMES 

3206 

69288 

72 . 16 

1 . 35 

463.25 

3.47 

67 . 77 

2.24 


NEVER OR ALMOST NEVER 

184 

4211 

4.38 

.69 

439.24 

7.56 

62.70 

4 . 18 

Belgium (French) 

OFTEN 

1029 

13472 

30 . 08 

. 76 

524.35 

2 . 95 

63 . 18 

1 . 93 


SOMETIMES 

2263 

29978 

66.93 

. 72 

501.06 

3.16 

64.03 

2 . 02 


NEVER OR ALMOST NEVER 

91 

1343 

3.00 

.39 

482.03 

10.29 

65.78 

5.48 
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2.7 PIRLS/prePIRLS Analyses with Teacher-level Data 


Analyses with teacher background data seek to make statements about students whose teachers have 
a given characteristic, rather than about teachers with a given characteristic. As our example of an 
analysis using teacher background data, we will investigate the percentage of students according to 
the years of experience of their teachers. The results of such an analysis are presented in Exhibit 7.3 
of PIRLS 2011 International Results in Reading and are reproduced here in Exhibit 2.20. 

We will use the Percentages and Means statistic type and select the Use PVs option to estimate 
the percentages of students, with their average reading achievement, by reporting categories of 
teachers’ years of experience. 

As with the previous examples, we first proceed to identify the variables relevant to the analysis 
in the appropriate files, and review the documentation for any specific national adaptations to the 
questions of interest. Because we are using a teacher-level variable, we need to look in the teacher 
background data files for the variable that contains the information on the years of experience of 
teachers. The variable ATBG01 contains information on teachers’ years of experience. That 
information was collapsed into reporting categories and stored in the derived variable ATDG01. 5 

The merged data file ATGALLR3 will be used for this example teacher-level analysis, which 
will be performed by the analysis module of the IE A IDB Analyzer using the following steps: 

1 . Open the Analysis Module of the IEA IDB Analyzer. 

2. Select the merged data file ATGALLR3 as the Analysis File. 

3. Select PIRLS (Using Teacher Weights) as the Analysis Type. 

4. Select Percentages and Means as the Statistic Type. 

5. Select Use PVs as the Plausible Value Option. 

6. Specify the variable ATDG01 as a second grouping variable and activate it by clicking the 
Grouping Variables field. Select ATDG01 from the list of available variables and move it to the 
Grouping Variables field by clicking the right arrow ( ► ) button in this section. 

7. Specify the achievement scores to be used for the analysis and activate them by clicking the 
Plausible Values field. Select ASRREA01-05 from the list of available variables and move it to 
the Plausible Values field by clicking the right arrow ( ► ) button in this section. 

8. The Weight Variable is selected automatically by the software; TCHWGT is selected by default 
because this example analysis uses teacher background data. 

9. Specify the name and the folder of the output files in the Output Files field by clicking the 
Define/Modify button. 

10. Click the Start SPSS button to create the SPSS syntax file and open it in an SPSS syntax 
window. The syntax file can be executed by opening the Run menu of SPSS and selecting the 
All menu option. If necessary, the IEA IDB Analyzer will display a prompt to confirm the 
overwriting of existing files. 


5 Supplement 3 to this User Guide describes the construction of the teacher-level derived variable ATDG01 . 
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Exhibit 2.20: Exhibit of Example Teacher-level Analysis, Taken from 
PIRLS 2011 International Results in Reading (Exhibit 7.3) 


Exhibit 7.3: Teachers' Years of Experience 

Reported by Teachers 

20 Years or More 


PIRLS 20111 


At Least 10 but Less 
than 20 Years 


At Least 5 but Less 
than 10 Years 


Less than 5 Years 


Average 




Percent of 
Students 


Average 

Achievement 


Percent of 
Students 


Average 

Achievement 


Percent of 
Students 


Average 

Achievement 

Percent of 
Students 


Average 

Achievement 

Experience 

Australia 

r 

42 (4.0) 


530 (5.1) 


22(3.5) 


533 (5.7) 


19 (2.9) 


529 (6.3) 

17(3.2) 


534 (7.2) 

17 (1.0) 

Austria 

■1 

55 (2.8) 

532 (2.4) 

25(2.7) 

526 (4.0) 

II 

11(1.9) 

II 

532 (3.9) 

10 (1.9) 

II 

513 (6.8) 

21 (0.6) 

Azerbaijan 


60(4.5) 


466 (3.8) 


26 (3.1) 


456 (8.6) 


11 (2.7) 


440 (10.4) 

4(2.0) 


484 (33.4) 

23 (1.1) 

Belgium (French) 

■1 

40 (3.9) 

II 

516 (3.9) 

II 

32 (3.8) 

II 

502 (4.9) 

II 

16(3.4) 

II 

504 (6.7) 

12 (2.5) 

II 

498 (12.5) 

16 (0.7) 

Bulgaria 


72 (3.3) 


533 (5.2) 


24 (3.0) 


532 (7.8) 


2 (1.0) 



2 (1.2) 



24 (0.6) 

Canada 

■1 

30 (2.2) 

II 

546 (2.5) 

II 

33 (2.6) 

II 

552 (3.7) 

II 

25 (1.9) 

II 

547 (3.4) 

12 (1.3) 

II 

545 (4.3) 

14 (0.4) 

Chinese Taipei 


26(3.3) 


558 (3.9) 


50 (3.8) 


551 (2.5) 


17(3.3) 


559 (5.1) 

7 (2.0) 


536 (6.1) 

15 (0.6) 

Colombia 

■1 

43 (4.5) 

II 

447 (6.0) 

II 

35 (4.4) 

II 

442 (5.9) 

II 

16(3.3) 

II 

451 (15.8) 

7(1.6) 

II 

496 (19.4) 

18 (0.7) 

Croatia 


56(3.4) 


557 (2.5) 


30 (2.9) 


545 (3.2) 


9(2.0) 


559 (6.1) 

5 (1.4) 


552 (6.5) 

21 (0.7) 

Czech Republic 


48 (4.0) 


543 (3.3) 

II 

27(3.6) 

II 

544 (3.5) 

II 

12 (2.4) 

II 

551 (5.4) 

13 (2.9) 

II 

554(7.7) 

18 (0.8) 

Denmark 


35 (3.7) 


557 (2.4) 


25 (3.1) 


552 (3.6) 


22 (2.6) 


554 (2.5) 

18 (2.8) 


553 (4.5) 

16 (0.9) 

England 


14(3.1) 


566 (8.0) 


27 (3.7) 

II 

550 (6.3) 

II 

29(3.7) 

II 

558 (5.0) 

30 (4.1) 

II 

538(5.7) 

10(0.7) 

Finland 


40 (3.1) 


567 (3.1) 


35 (3.1) 


570 (2.5) 


12 (2.0) 


571 (4.4) 

13 (2.0) 


564 (4.2) 

17 (0.6) 

France 

■1 

34 (3.4) 

II 

530 (3.7) 

II 

36 (3.0) 

II 

516 (3.8) 

II 

19(2.5) 

II 

520 (3.5) 

11 (2.1) 

II 

506 (8.0) 

16 (0.7) 

Georgia 


59 (3.4) 


486 (3.3) 


29 (3.5) 


489 (6.2) 


8(1.9) 


497 (18.8) 

4(1.6) 


471 (16.4) 

22 (0.7) 

Germany 

■l 

44 (3.8) 

II 

539 (3.6) 

II 

25 (3.2) 

II 

542 (4.8) 

II 

14(2.7) 

II 

546 (6.4) 

16 (2.6) 

II 

540 (5.4) 

19 (0.9) 

Flong Kong SAR 


21 (3.5) 


564 (6.4) 


53 (4.0) 


569 (3.8) 


15 (3.3) 


573 (5.9) 

12 (2.5) 


582 (5.1) 

14 (0.7) 

Hungary 


73 (3.0) 


544 (3.1) 

II 

16 (2.8) 

II 

525 (12.8) 

II 

7(1.7) 

II 

537 (11.7) 

4(1.4) 

II 

505 (10.9) 

24 (0.6) 

Indonesia 

r 

52 (4.9) 


438 (5.4) 


16(3.7) 


432 (10.3) 


19 (3.8) 


429 (10.6) 

12(3.1) 


395 (15.3) 

18 (1.0) 

Iran, Islamic Rep. of 

■1 

41 (3.6) 

II 

476 (5.3) 

II 

41 (3.5) 

II 

449 (5.0) 

II 

10 (1.9) 

II 

447 (13.3) 

9(1.8) 

II 

421 (10.8) 

17 (0.6) 

Ireland 


24 (3.0) 


551 (4.5) 


21 (3.1) 


555 (5.7) 


27 (3.3) 


550 (4.0) 

27 (2.9) 


553 (4.6) 

12 (0.7) 

Israel 

■1 

30 (3.8) 

II 

545 (5.7) 

II 

36 (3.9) 

II 

546 (7.0) 

II 

16(2.7) 

II 

537 (9.6) 

18 (3.0) 

II 

531 (11.3) 

15 (0.8) 

Italy 


69 (3.6) 


543 (2.8) 


24 (3.4) 


539 (3.4) 


6(1.7) 


539 (7.4) 

1 (0.8) 


~~ 

24 (0.7) 

Lithuania 

■1 

71 (2.5) 

II 

527 (2.6) 

II 

26(2.3) 

II 

534 (3.6) 

II 

2 (1.0) 

II 

— 

1 (0.5) 

II 

■■I 

24(0.6) 

Malta 


18 (0.1) 


491 (2.8) 


38 (0.1) 


475 (2.5) 


29 (0.1) 


472 (2.4) 

15 (0.1) 


477 (4.1) 

12 (0.0) 

Morocco 

■1 

56 (4.2) 

II 

309 (5.3) 

II 

31 (4.4) 

II 

294 (8.9) 

II 

8(1.6) 

II 

336(17.4) 

5 (1.4) 

II 

394 (16.4) 

21 (0.6) 

Netherlands 


28 (3.1) 


549 (3.0) 


29 (3.4) 


542 (3.2) 


24(3.2) 


549 (3.4) 

19 (3.0) 


545 (5.3) 

14 (0.8) 

New Zealand 

■1 

20 (2.5) 

II 

542 (5.3) 

II 

26 (2.6) 

II 

533 (5.1) 

II 

28(2.5) 

II 

540 (4.6) 

27(2.5) 

II 

521 (5.4) 

11 (0.6) 

Northern Ireland 

r 

34 (4.7) 


556 (3.8) 


36 (4.0) 


563 (4.8) 


24 (4.2) 


561 (6.2) 

7(2.3) 


564 (20.2) 

16 (1.0) 

Norway 

■1 

31 (4.4) 

II 

506 (4.1) 

II 

40 (4.6) 

II 

510 (2.7) 

II 

15 (3.3) 

II 

507 (4.9) 

15 (2.4) 

II 

509 (5.9) 

16 (1.0) 

Oman 


12 (1.9) 


409 (6.7) 


36 (2.8) 


3% (4.6) 


31 (2.9) 


385 (5.2) 

21 (2.1) 


385 (6.4) 

11 (0.4) 

Poland 

■1 

83 (2.2) 

II 

526 (2.4) 

II 

11 (2.1) 

II 

529 (7.4) 

II 

4(1.5) 

II 

518 (10.7) 

2 (0.9) 

II 

- M 

23 (0.4) 

Portugal 


36 (3.2) 


549 (3.4) 


46 (3.6) 


532 (4.2) 


14(2.5) 


535 (6.5) 

4(1.6) 


566 (11.9) 

17 (0.6) 

Qatar 

■1 

20 (4.3) 

II 

450 (14.6) 

II 

25 (4.3) 

II 

447 (9.1) 

II 

30 (4.1) 

II 

422 (8.6) 

25 (3.5) 

II 

388 (9.0) 

11 (0.7) 

Romania 


57 (3.7) 


511 (4.8) 


31 (3.5) 


487 (8.2) 


9(2.3) 


478 (13.4) 

2 (1.0) 



23 (0.8) 

Russian Federation 

■1 

73 (3.0) 

II 

571 (3.0) 

II 

22 (2.7) 

II 

567 (6.3) 

II 

3 (1.1) 

II 

526 (16.2) 

3 (1.5) 

II 

559 (12.0) 

24 (0.7) 

Saudi Arabia 


16 (3.1) 


422 (14.5) 


50 (4.5) 


439 (5.3) 


18 (3.0) 


428 (15.2) 

17(3.5) 


412 (12.9) 

13 (0.7) 

Singapore 

■1 

17 (1.8) 

II 

570 (7.4) 

II 

27(2.6) 

II 

563 (6.9) 

II 

24(2.5) 

II 

575 (5.5) 

32(2.1) 

II 

564(5.7) 

11 (0.5) 

Slovak Republic 


55 (3.0) 


536 (4.4) 


28 (2.9) 


531 (3.4) 


10(2.1) 


546 (5.9) 

7(1.8) 


530 (8.4) 

20 (0.5) 

Slovenia 

■1 

57 (3.8) 

II 

532 (2.3) 

II 

27(3.1) 

II 

532 (3.9) 

II 

10(2.2) 

II 

517(6.4) 

6(1.5) 

II 

523 (8.0) 

21 (0.7) 

Spain 


59(4.1) 


519 (3.0) 


19 (3.4) 


502 (7.1) 


8(1.3) 


510 (7.3) 

14(2.6) 


502 (6.9) 

21 (0.8) 

Sweden 

■1 

29(4.2) 

II 

546 (3.8) 

II 

45 (4.0) 

II 

543 (3.8) 

II 

18(2.7) 

II 

529 (4.3) 

8(1.9) 

II 

551 (6.3) 

16(0.9) 

Trinidad and Tobago 


43 (4.0) 


487 (6.5) 


35 (4.0) 


459 (7.1) 


9(2.4) 


459 (15.4) 

12(2.5) 


452 (13.8) 

18 (1.0) 

United Arab Emirates 

■1 

12 (1.6) 

II 

434 (9.1) 

II 

30(2.3) 

II 

441 (7.3) 

II 

33 (2.3) 

II 

439 (6.2) 

25 (2.0) 

II 

436 (6.0) 

10(0.3) 

United States 


28 (2.2) 


569 (3.8) 


38(2.1) 


553 (3.1) 


19 (2.0) 


550 (4.3) 

15 (1.9) 


552 (5.2) 

14 (0.5) 

International Avg. 


41 (05) 


517 (0.8) 


31 (0.5) 


511 (0.9) 


16 (0.4) 


510(1.4) 

12 (03) 


507 (1.7) 

17 (0.1) 

Sixth Grade Participants 

Botswana 


29 (4.0) 


434 (9.2) 


32 (4.1) 


421 (10.4) 


23 (3.7) 


401 (8.4) 

16 (3.0) 


420 (9.8) 

14 (0.8) 

Honduras 

■1 

29 (4.2) 

II 

465 (6.4) 

II 

37 (4.6) 

II 

436 (7.9) 

II 

17 (3.7) 

II 

458 (7.2) 

17 (4.0) 

II 

459 (20.6) 

14 (0.9) 

Kuwait 

s 

9(2.9) 


419 (7.2) 


23 (4.5) 


412 (16.5) 


16 (3.8) 


429 (21.7) 

52(4.9) 


419 (12.8) 

7(0.7) 

Morocco 

r 

53 (4.4) 

II 

422 (7.4) 

II 

38(4.3) 

II 

407 (9.3) 

II 

7 (2.0) 

II 

468 (14.8) 

2 (0.9) 

II 

— 

21 (0.7) 

Benchmarking Participants 0 

Alberta, Canada 


33 (3.8) 


544 (4.2) 


23 (3.2) 


560 (5.8) 


23 (3.6) 


554 (6.2) 

21 (3.1) 


533 (5.5) 

14 (0.8) 

Ontario, Canada 

■1 

20 (2.9) 

II 

549 (6.3) 

II 

31 (4.0) 

II 

550 (5.1) 

II 

33 (3.5) 

II 

553 (4.9) 

17(2.7) 

II 

551 (7.3) 

12 (Oi) 

Quebec, Canada 


33 (4.2) 


538 (3.7) 


40 (4.6) 


539 (3.2) 


20 (3.6) 


533 (6.7) 

7(1.9) 


537 (6.4) 

16 (0.7) 

Maltese - Malta 

■1 

17(0.1) 

II 

467 (3.7) 

II 

35 (0.1) 

II 

456 (2.3) 

II 

34 (0.1) 

II 

454(2.6) 

14 (0.1) 

II 

460 (3.7) 

12(0.0) 

Eng/Afr (5) - RSA 


43 (5.0) 


415 (15.3) 


24 (4.6) 


420 (14.8) 


15 (3.5) 


450 (25.3) 

17(4.7) 


455 (22.5) 

17(1.1) 

Andalusia, Spain 

■1 

58 (3.9) 

II 

521 (3.2) 

II 

16(3.1) 

II 

520 (5.7) 

II 

11 (2.6) 

II 

501 (8.7) 

15 (2.8) 

II 

498 (7.2) 

21 (1.0) 

Abu Dhabi, UAE 


15(3.2) 


397 (12.8) 


31 (4.4) 


428 (13.5) 


28 (4.1) 


419 (10.4) 

26(3.7) 


438 (9.7) 

10 (0.6) 

Dubai, UAE 

r 

13 (2.6) 

II 

492 (10.3) 

II 

28 (3.6) 

II 

477 (9.7) 

II 

38 (4.3) 

II 

482 (8.8) 

21 (2.6) 

II 

470(8.5) 

11 (0.6) 

Florida, US 

r 

26(4.3) 


572 (8.6) 


33 (4.3) 


576 (6.1) 


30 (3.9) 


566 (6.6) 

11 (3.5) 


563 (8.0) 

15 (0.9) 


0 Republic of South Africa (RSA) tested 5th grade students receiving instruction in English (ENG) or Afrikaans (AFR). 

( ) Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 

A tilde R Indicates Insufficient data to report achievement. 

An "r" Indicates data are available for at least 70% but less than 85% of the students. An “s” indicates data are available for at least 50% but less than 70% of the students. 
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Exhibit 2.21 shows the completed analysis module for this example analysis, and Exhibit 2.22 
displays the results. 

Exhibit 2.21 : IDB Analyzer Setup for Example Teacher-level Analysis 


f IEA IDB Analyzer Analysis Module - (Version 3.0.47) 


O Analysis File: | C:\PIRLS2011\Data\ATGALLR3.sav | Select 


0 

Analysis Type: 

Statistic Type: 

Plausible Value Option: Benchmark Option: 

Missing Data Option: Number of Decimals: 


| PIRLS (Using Teacher Weights) 

* | | Percentages and Means » 

| [Use PVs 

' |2 H 








Q Select Variables: 




O Output Files: C:\PIRLS2011\Data\REAbyATDG01.* 


| Modify | 
O [ Start SPSS ] 


Return to Main Menu 


Help | 


PIRLS (Using Teacher Weights) Percentages and Means Use PVs 


Exhibit 2.22: Output for Example T eacher-level Analysis 


Average for ASRREAO by 
* COUNTRY ID* 

IDCNTRY ATDG01 

* TEACHERS YEARS OF EXPERIENCE* 

N of 
Cases 

Sum of 
TCHWGT 

Percent 

Percent 
(s.e. ) 

ASRREAO 

(Mean) 

ASRREAO 

(s.e.) 

Std.Dev 

Std.Dev. 
(s.e. ) 

Australia 

20 YEARS OR MORE 

1954 

83004 

41.50 

3 . 98 

529 . 67 

5.13 

81.03 

2 . 71 


AT LEAST 10 BUT LESS THAN 20 YEARS 

1066 

44750 

22.37 

3.52 

533.03 

5.68 

79 . 87 

3.16 


AT LEAST 5 BUT LESS THAN 10 YEARS 

1000 

38063 

19 . 03 

2 . 88 

529 . 09 

6.32 

80 . 79 

4 . 08 


LESS THAN 5 YEARS 

868 

34188 

17.09 

3.25 

533 . 76 

7.24 

77.44 

3.34 

Austria 

20 YEARS OR MORE 

2445 

42361 

54 . 79 

2.83 

532.39 

2.42 

62 . 01 

1.22 


AT LEAST 10 BUT LESS THAN 20 YEARS 

1161 

19233 

24 . 87 

2 . 73 

525 . 88 

4 . 04 

64.94 

2.21 


AT LEAST 5 BUT LESS THAN 10 YEARS 

523 

8339 

10.78 

1.93 

532.30 

3.93 

60.56 

3 . 77 


LESS THAN 5 YEARS 

558 

7389 

9.56 

1.88 

512.69 

6 . 76 

66.57 

2.46 

Azerbaijan, Republic of 

20 YEARS OR MORE 

2490 

53345 

59.79 

4.51 

466.47 

3 . 77 

63.09 

1 . 70 


AT LEAST 10 BUT LESS THAN 20 YEARS 

1139 

23144 

25 . 94 

3.11 

456.15 

8.56 

71 . 67 

4 . 75 


AT LEAST 5 BUT LESS THAN 10 YEARS 

349 

9506 

10 . 65 

2 . 68 

439 . 81 

10.43 

67.26 

4.47 


LESS THAN 5 YEARS 

121 

3231 

3.62 

1.95 

483 . 72 

33.44 

86.08 

19.66 

Belgium (French) 

20 YEARS OR MORE 

1445 

19025 

40.39 

3 . 88 

516 . 19 

3 . 91 

61 . 53 

1 . 80 


AT LEAST 10 BUT LESS THAN 20 YEARS 

1085 

15154 

32 . 17 

3.83 

501.83 

4.95 

63.61 

2.54 


AT LEAST 5 BUT LESS THAN 10 YEARS 

534 

7413 

15 . 74 

3.45 

503.82 

6 . 72 

61 . 75 

3 . 05 


LESS THAN 5 YEARS 

431 

5507 

11 . 69 

2.53 

498.39 

12.47 

75 . 68 

6.43 
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Each country’s results are displayed on four lines, one for each value of the ATDG01 variable. 
The results are presented in the same manner as in the previous examples, with countries identified 
in the first column and the second column describing the categories of the analysis variable 
ATDG01. 

As shown in the first four lines of results, 41.50 percent of students in Australia were taught by 
teachers with 20 years or more of experience, 22.37 percent were taught by teachers with 10 to 19 
years of experience, 19.03 percent by teachers with 5 to 9 years of experience, and 17.09 percent by 
teachers with less than 5 years of experience. Also, the estimated average reading achievement 
was 529.67 (standard error of 5.13) for students taught by teachers with 20 years or more of 
experience, 533.03 (standard error of 5.68) for students taught by teachers with 10 to 19 years of 
experience, 529.09 (standard error of 6.32) for students taught by teachers with 5 to 9 years of 
experience, and 533.76 (standard error of 7.24) for students taught by teachers with less than 5 years 
of experience. The IEA IDB Analyzer also produces the standard deviations of achievement for all 
subgroups of ATDG01. 

2.8 PIRLS/prePIRLS Analyses with School-level Data 

When performing analyses with school background data, the data are analyzed to make statements 
about students attending schools with a given characteristic, rather than about schools with a given 
characteristic. Our example of an analysis using school background data will compute the 
percentages of students, with their average reading achievement, who attended schools composed of 
students with different levels of economic background. The results of this analysis are presented in 
Exhibit 5.2 of PIRLS 2011 International Results in Reading and are replicated here in Exhibit 2.23. 

We will use the Percentages and Means statistic type and select the Use PVs option to estimate 
the percentages of students, with their average reading achievement, by reporting categories of 
students’ economic background as reported by school principals. 

The information for this analysis is found in the school-level derived variable ACDG03, where 
schools are characterized as being composed of more affluent students than disadvantaged students, 
composed of more disadvantaged students than affluent students, or composed of neither more 
affluent students nor more disadvantaged students. 6 

The merged data file ACGALLR3 will be used for this example school-level analysis and it is 
performed by the analysis module of the IEA IDB Analyzer using the following steps: 

1 . Open the Analysis Module of the IEA IDB Analyzer. 

2. Select the merged data file ACGALLR3 as the Analysis File. 

3. Select PIRLS (Using Student Weights) as the Analysis Type. 

4. Select Percentages and Means as the Statistic Type. 

5. Select Use PVs as the Plausible Value Option. 


6 Supplement 3 to this User Guide describes the construction of the school-level derived variable ACDG03 from responses to questions posed to the 
school principals. 
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Exhibit 2.23: Exhibit of Example School-level Analysis, Taken from 
PIRLS 2011 International Results in Reading (Exhibit 5.2) 

Exhibit 5.2: School Composition by Student Economic Background 

Reported by Principals 


PIRLS 20111 




More Affluent 

- Schools Where More 





More Disadvantaged 

-Schools Where 



than 25% of Students Come from 


Neither More Affluent nor 
More Disadvantaged 


More than 25% of Students Come 




Economically Affluent Homes and Not 



from Economically Disadvantaged 


Country 


More than 25% from Economically 



Homes and Not More than 25% from 




Disadvantaged Homes 






Economically Affluent Homes 




Percent 


Average 


Percent 


Average 


Percent 


Average 




of Students 


Achievement 


of Students 


Achievement 


of Students 


Achievement 


Australia 


32 (3.9) 


556 (3.9) 


41 (4.0) 


526 (3.6) 


27 (3.4) 


500 (5.7) 


Austria 

■1 

31 (4.0) 

■■ 

539 (2.9) 

II 

48 (3.8) 

II 

532 (2.4) 

■1 

21 (3.9) 

■I 

505 (4.4) 

1 

Azerbaijan 

r 

11 (2.5) 


464 (10.5) 


32(4.7) 


470 (8.1) 


57 (4.9) 


462 (6.1) 


Belgium (French) 


49(5.5) 

■I 

522 (3.1) 

II 

27(4.5) 

■I 

501 (6.4) 

■1 

25(4.5) 

■I 

483 (4.6) 

I 

Bulgaria 


15(3.3) 


561 (6.2) 


43 (4.3) 


549 (4.5) 


42 (4.4) 


506 (7.8) 


Canada 

■1 

39(2.4) 

■I 

557 (3.0) 

II 

34 (2.9) 

■I 

549 (2.7) 

■1 

28 (2.6) 

■I 

533 (2.6) 

1 

Chinese Taipei 


22 (3.3) 


563 (4.5) 


67 (3.5) 


554 (2.4) 


11 (2.0) 


525 (6.8) 


Colombia 

■1 

7(2.0) 

■I 

521 (17.1) 

II 

15 (3.4) 

■■ 

471 (11.7) 

■1 

78 (3.9) 

■I 

432 (4.7) 

1 

Croatia 


38 (4.0) 


560 (3.1) 


38 (4.2) 


550 (2.3) 


24(3.2) 


551 (4.9) 


Czech Republic 

■1 

37(3.7) 

■I 

551 (3.2) 

II 

46 (4.4) 

II 

548 (22) 

■1 

17 (3.1) 

■I 

524 (6.7) 

1 

Denmark 


60 (3.6) 


561 (2.1) 


33 (3.3) 


546 (3.0) 


7(1.8) 


524 (7.7) 


England 

■1 

32 (4.8) 

■I 

568 (4.9) 

II 

33 (4.9) 

■■ 

554 (4.0) 

■1 

35 (4.0) 

■I 

527 (4.7) 

1 

Finland 


43 (4.2) 


576 (2.4) 


47 (4.3) 


567 (2.7) 


10 (2.6) 


541 (4.0) 


France 

■1 

37(4.3) 

■I 

539 (3.5) 

II 

35 (3.9) 

■■ 

522 (4.6) 

■1 

28(3.7) 

■I 

493 (4.7) 

1 

Georgia 


16 (3.0) 


496 (8.8) 


41 (4.3) 


494 (5.9) 


43 (4.0) 


480 (4.5) 


Germany 


21 (2.8) 


555 (3.3) 

II 

53 (3.7) 

II 

549 (3.0) 

■1 

26(3.3) 

■I 

512 (5.5) 

1 

Flong Kong SAR 


20 (3.3) 


580 (3.9) 


30 (4.7) 


569 (5.3) 


50 (4.7) 


568 (4.4) 


Flungary 

■1 

21 (3.6) 

■I 

573 (6.3) 

II 

31 (4.3) 

II 

557 (4.2) 

■1 

48 (4.0) 

■I 

516 (5.2) 

1 

Indonesia 

r 

20 (4.1) 


475 (5.6) 


21 (3.9) 


431 (7.7) 


59(4.6) 


421 (6.0) 


Iran, Islamic Rep. of 

27 (3.6) 

■I 

488 (7.6) 

II 

27 (4.1) 

■I 

460 (6.7) 

■1 

46 (4.2) 

■I 

438 (4.8) 

1 

Ireland 

r 

39 (4.7) 


568 (3.3) 


30 (4.2) 


554 (4.7) 


31 (3.4) 


523 (4.0) 


Israel 

r 

35 (3.6) 

■I 

566 (6.1) 

II 

28 (3.4) 

■I 

559 (4.9) 

■1 

37 (3.4) 

■I 

500 (6.5) 

1 

Italy 


37 (3.8) 


541 (4.5) 


43 (3.7) 


545 (3.6) 


20 (2.9) 


531 (5.0) 


Lithuania 

■1 

19(3.3) 

■■ 

552 (5.8) 

II 

43 (4.6) 

II 

529 (3.3) 

■1 

38 (3.5) 

■I 

518 (3.1) 

1 

Malta 


47 (0.1) 


482 (2.2) 


43 (0.1) 


478 (2.4) 


10 (0.1) 


421 (5.3) 


Morocco 

■1 

12(2.1) 

■I 

372 (16.7) 

II 

13 (2.8) 

■I 

317(11.6) 

■1 

75 (3.3) 

■I 

304 (6.1) 

1 

Netherlands 

r 

63 (4.9) 


553 (2.3) 


23 (3.9) 


544 (2.6) 


15 (3.8) 


522 (8.2) 


New Zealand 

■1 

39 (3.4) 

■I 

560 (3.2) 

II 

34 (3.6) 

■■ 

533 (3.7) 

■1 

27 (2.5) 

■I 

489 (4.2) 

1 

Northern Ireland 

r 

36(4.7) 


578 (4.9) 


38(4.3) 


555 (3.3) 


26 (3.8) 


534 (5.8) 


Norway 

■1 

53 (5.3) 

■■ 

511 (3.3) 

II 

44 (5.3) 

■■ 

505 (2.9) 

■1 

3 (1.2) 

■I 

488 (16.9) 

1 

Oman 

r 

44 (3.4) 


3% (4.3) 


25 (2.9) 


378 (6.7) 


31 (2.9) 


370 (5.1) 


Poland 

■1 

8(2.1) 

■I 

536 (10.2) 

II 

61 (3.8) 

■I 

532 (2.9) 

■1 

31 (3.7) 

■I 

512 (3.5) 

1 

Portugal 


30 (4.5) 


552 (4.0) 


39 (4.9) 


547 (4.1) 


31 (4.8) 


522 (4.6) 


Qatar 

II 

68 (3.0) 

■I 

423 (5.0) 

II 

21 (2.3) 

■I 

441 (10.6) 

■1 

11 (1.9) 

■I 

378 (7.5) 

1 

Romania 


19(3.1) 


538 (9.2) 


24 (4.0) 


507 (8.8) 


57(4.8) 


491 (6.5) 


Russian Federation 

■1 

58 (3.2) 

■I 

576 (4.0) 

II 

29(3.3) 

■■ 

562 (4.8) 

■1 

13 (2.1) 

■I 

549 (8.5) 

1 

Saudi Arabia 

r 

42 (4.7) 


445 (8.5) 


30 (4.3) 


439 (6.0) 


29 (4.0) 


408 (10.0) 


Singapore 

■1 

40 (0.0) 

■I 

590 (5.2) 

II 

50 (0.0) 

■■ 

556 (4.5) 

■1 

10 (0.0) 

■I 

541 (14.3) 

1 

Slovak Republic 


24(3.3) 


551 (3.9) 


56 (3.4) 


542 (2.5) 


20(3.2) 


499 (8.0) 


Slovenia 

■1 

42 (4.0) 

■I 

533 (3.7) 

II 

40 (4.0) 

■I 

531 (2.7) 

■1 

18 (3.0) 

■I 

521 (6.7) 

1 

Spain 


51 (3.7) 


527 (4.0) 


31 (3.3) 


511 (4.7) 


18(3.1) 


482 (5.1) 


Sweden 

r 

76 (4.2) 

■■ 

547 (2.8) 

II 

17 (4.1) 

■I 

532 (7.2) 

■1 

7 (1.5) 

■I 

509 (8.5) 

1 

Trinidad and Tobago 


20 (3.2) 


508 (7.5) 


26 (4.0) 


464 (9.2) 


54 (4.2) 


460 (5.7) 


United Arab Emirates 

r 

68(2.2) 

■■ 

440 (3.6) 

II 

20 (1.6) 

■■ 

444 (6.2) 

■1 

12(1.7) 

■I 

412 (5.5) 

1 

United States 

r 

18 (2.2) 


591 (2.9) 


31 (2.6) 


570 (3.5) 


51 (2.3) 


537 (2.4) 


International Avg. 


35 (0.5) 


530 (0.9) 


35 (0.6) 


515 (0.8) 


30 (0.5) 


490 (1.0) 


Sixth Grade Participants 

Botswana 


32 (3.6) 


454 (7.9) 


25 (4.0) 


403 (8.3) 


43 (4.3) 


390 (4.3) 


Honduras 

■1 

16 (4.0) 

■■ 

518 (14.5) 

II 

13 (3.8) 

■■ 

440 (14.3) 

■1 

71 (4.9) 

■I 

444 (5.7) 

1 

Kuwait 

r 

30 (4.5) 


429 (13.9) 


35 (4.9) 


431 (14.1) 


35(5.3) 


402 (11.2) 


Morocco 

s 

12(2.3) 

■■ 

465 (16.5) 

II 

12 (2.6) 

■I 

456 (11.1) 

■1 

76(3.1) 

■I 

415 (5.8) 

1 

Benchmarking Participants 0 

Alberta, Canada 


35(3.7) 


564 (4.8) 


40 (4.3) 


550 (4.3) 


25 (3.8) 


527(5.5) 


Ontario, Canada 

■l 

32 (4.7) 

■■ 

557 (5.4) 

II 

34(5.3) 

■■ 

555 (4.3) 

■1 

34 (5.4) 

■I 

537 (4.4) 

1 

Quebec, Canada 


60 (4.1) 


544 (2.6) 


25 (4.0) 


526 (5.2) 


15(2.7) 


528 (4.9) 


Maltese - Malta 

■1 

47 (0.1) 

■■ 

459 (2.1) 

II 

43 (0.2) 

■■ 

467 (2.6) 

■1 

10 (0.1) 

■I 

419 (4.5) 

1 

Eng/Afr (5) - RSA 

r 

22(4.1) 


507 (15.1) 


23 (6.1) 


419 (20.4) 


55 (6.7) 


382 (13.5) 


Andalusia, Spain 

■1 

47 (4.3) 

■■ 

525 (3.9) 

II 

34(3.5) 

■■ 

519 (3.2) 

■1 

19(3.7) 

■I 

490 (5.9) 

1 

Abu Dhabi, UAE 

s 

75 (4.5) 


423 (7.4) 


12(3.2) 


422 (18.7) 


13 (3.5) 


402 (10.7) 


Dubai, UAE 

■1 

67 (0.4) 

■■ 

473 (2.8) 

II 

22(0.3) 

■■ 

498 (4.3) 

■1 

11 (0.2) 

■I 

416 (5.0) 

1 

Florida, US 

r 

11 (4.6) 


598 (5.1) 


20 (4.7) 


590 (9.1) 


68 (4.7) 


559 (4.5) 



0 Republic of South Africa (RSA) tested 5th grade students receiving instruction in English (ENG) or Afrikaans (AFR). 


( ) Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 

An "r" indicates data are available for at least 70% but less than 85% of the students. An "s” indicates data are available for at least 50% but less than 70% of the students. 
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SOURCE: lEA's Progress in International Reading Literacy Study - PIRLS 201 1 




6. Specify the variable ACDG03 as a second grouping variable and activate it by clicking the 
Grouping Variables field. Select ACDG03 from the list of available variables and move it to the 
Grouping Variables field by clicking the right arrow ( ► ) button in this section. 

7. Specify the achievement scores to be used for the analysis and activate them by clicking the 
Plausible Values field. Select ASRREA01-05 from the list of available variables and move it to 
the Plausible Values field by clicking the right arrow ( ► ) button in this section. 

8. The Weight Variable is selected automatically by the software; TOTWGT is selected by default 
because this example analysis uses school background data linked to student background data. 

9. Specify the name and the folder of the output files in the Output Files field by clicking the 
Define/Modify button. 

10. Click the Start SPSS button to create the SPSS syntax file and open it in an SPSS syntax 
window. The syntax file can be executed by opening the Run menu of SPSS and selecting the 
All menu option. If necessary, the IEA IDB Analyzer will display a prompt to confirm the 
overwriting of existing files. 

Exhibit 2.24 shows the completed analysis module for this example analysis and the results are 

presented in Exhibit 2.25. 

Exhibit 2.24: IDB-Analyzer Setup for Example School-level Analysis 


F IEA IDB Analyzer Analysis Module - (Version 3.0.47) 


O Analysis File: | C:\PIRLS2011\Pata\ACGALLR3.sav ( Select ] 


0 

Analysis Type: 

Statistic Type: 

Plausible Value Option: Benchmark Option: 

Missing Data Option: Number of Decimals: 


| PIRLS (Using Student Weights) 

w | | Percentages and Means w 

| | Use PVs ’’’I | 

• [2 -I 








Q Select Variables: 


1 A Grouping Variables: [3 Exclude Missing From Analysis 



Name 

Description 


rnrn 

► if IDCNTRY 

♦COUNTRY ID* 



4$ACDG03 

♦SCHOOL COMPOSITION BY STUDENT BACKGD* 


(v) Analysis Variables: 
A Plausible Values: 


fiTG 

Name 

Description 


| AASRREA01-05 

1ST TO 5TH PLAUSIBLE VALUE: OVERALL READING P... ' | 


v i Dependent Variable: 
Weight Variable: 



Name 

Description 



| ► <3$ TOTWGT 

TOTAL STUDENT WEIGHT* 


y 1 Achievement Benchmarks / Percentiles: 

L 



J 



J Output Files: C:\PIRLS2011\Data\REAbyACDG03* 


[ .Modify _ 

0 | Start SPSS 


| Return to Main Menu 1 1 Help | 


PIRLS (Using Student Weights) Percentages and Means Use PVs 
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Exhibit 2.25: Output for Example School-Level Analysis 


Average for ASRREA0 by 
* COUNTRY ID* 

IDCNTRY ACDG03 

* SCHOOL COMPOSITION BY STUDENT 

BACKGD* 

N of 
Cases 

Sum of 
TOTWGT 

Percent 
Percent (s.e.) 

ASRREA0 

(Mean) 

ASRREA0 
(s.e. ) 

std.Dev 
Std.Dev (s.e.) 

Australia 

MORE AFFLUENT 




1759 

71259 

32 . 08 

3 . 91 

555 . 68 

3 . 94 

72 . 03 

1 . 83 


NEITHER MORE AFFLUENT 

NOR 

MORE 

DISADVAN 

2425 

90918 

40.93 

3.99 

526.01 

3.57 

76.49 

2 . 14 


MORE DISADVANTAGED 




1162 

59962 

26 . 99 

3.42 

499.56 

5 . 70 

81 . 90 

3.22 

Austria 

MORE AFFLUENT 




1299 

21774 

30.53 

4 . 04 

539.43 

2 . 91 

60.44 

1.28 


NEITHER MORE AFFLUENT 

NOR 

MORE 

DISADVAN 

1961 

34354 

48.16 

3.79 

531.63 

2.40 

61.31 

1.55 


MORE DISADVANTAGED 




1041 

15199 

21.31 

3 . 91 

505.26 

4.44 

66.37 

2 . 78 

Azerbaijan, Republic of 

MORE AFFLUENT 




458 

9540 

11.34 

2.50 

463.52 

10.46 

61.83 

4.06 


NEITHER MORE AFFLUENT 

NOR 

MORE 

DISADVAN 

1271 

26743 

31 . 80 

4 . 71 

470 . 17 

8 . 10 

67 . 64 

3 . 81 


MORE DISADVANTAGED 




2368 

47824 

56.86 

4.88 

462 . 14 

6.10 

67.56 

3.10 

Belgium (French) 

MORE AFFLUENT 




1558 

20853 

48.55 

5.48 

522.40 

3 . 06 

59 . 89 

1.46 


NEITHER MORE AFFLUENT 

NOR 

MORE 

DISADVAN 

871 

11519 

26.82 

4.47 

501.01 

6.37 

63.39 

3.26 


MORE DISADVANTAGED 




743 

10577 

24.63 

4.49 

483.24 

4.60 

61.37 

1 . 92 


In this example, each country’s results are presented on three lines, one for each value of the 
ACDG03 variable. The results are presented in the same manner as in previous examples, with 
countries identified in the first column and the second column describing the categories of ACDG03. 

As shown in the first three lines of results, 32.08 percent of students in Australia attended schools 
with more affluent students than disadvantaged students (standard error of 3.91), 40.93 percent 
attended schools with neither more affluent students nor more disadvantaged students (standard error 
of 3.99), and 26.99 percent attended schools with more disadvantaged students than affluent students 
(standard error of 3.42). Also, the estimated average reading achievement was 555.68 (standard error 
of 3.94) for students in schools with more affluent students, 526.01 (standard error of 3.57) for 
students in schools with neither more affluent students nor more disadvantaged students, and 499.56 
(standard error of 5.70) for students in schools with more disadvantaged students. The IEA IDB 
Analyzer also produces the standard deviations of achievement for all subgroups of ACDG03. 
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Chapter 3 

Using SAS to Analyze the 

PIRLS 201 1 International Database 


3.1 Overview 


This chapter presents some basic examples of analyses that can be performed with the PIRLS 2011 
International Database using the SAS statistical analysis system (SAS Institute, 2012) and the SAS 
programs and macros provided. The SAS macros use sampling weights and the jackknifing 
algorithm to deal with the PIRLS complex sample design and take into account plausible values 
when analyzing student achievement. 

Although some familiarity with the structure of the PIRLS 2011 database will be helpful, the 
analyses presented in this chapter are simple in nature and are designed primarily to familiarize users 
with the various data files and their structure, as well as the variables to be used in most analyses. 
Chapter 4 provides a more detailed description of the data files contained in the International 
Database, including their structure and contents, along with detailed information on all the 
supporting documentation provided with the International Database. 

In general, the examples in this chapter compute percentages of students in specified subgroups, 
average reading achievement in those subgroups, and appropriate standard errors for these statistics. 
Additional examples compute regression coefficients and their standard errors. The example 
analyses, using student, home, teacher and school data, replicate some of the analyses that are 
included in PIRLS 2011 International Results in Reading (Mullis, Martin, Foy, & Drucker, 2012). 
Users are encouraged to practice analyzing the PIRLS 2011 data by replicating some of the exhibits 
presented in PIRLS 2011 International Results in Reading. 

Before conducting any statistical analyses with the PIRLS 201 1 International Database, users 
should download and copy the contents of the International Database either on their computer or on a 
server. 1 For the purposes of this chapter, we will assume all files have been copied to the folder titled 
“C:\PIRLS201 1\.” All SAS programs presented in this chapter are available in the International 
Database. They can be adapted to perform a variety of analyses with some basic knowledge of the 
SAS language. With a little experience and some practice with these programs, users should be able 
to make the necessary modifications to obtain the desired results. The example SAS programs invoke 
SAS macros that will be described in this chapter. Although users will be expected to modify the 
example programs, there is no need to make any changes within the SAS macros. 


1 The PIRLS 201 1 International Database is available on the PIRLS 201 1 International Database and User Guide webpage 

0 http://timssandpirls.bc.edu/pirls2011/international-database.html f) and on the IEA Study Data Repository website ( http://rms.iea-dpc.org / ). 
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3.2 SAS Programs and Macros 


The PIRLS 2011 International Database includes a number of SAS programs needed to process the 
SAS data files, compute survey results, and carry out example analyses. This chapter gives detailed 
instructions on how to adapt and make use of them. The following programs and macros are 
available: 

CONVERT.SAS 

This SAS program converts the SAS export fdes found in the International Database into SAS data 
files. All programs and macros described in this chapter require that the SAS export files be 
converted into SAS data files. 

ASASCRR3.SAS, ASASCRL1 .SAS 

These two SAS programs can be used to convert the response codes to the PIRLS and prePIRLS 
2011 achievement items to their corresponding score levels. 

JOIN.SAS 

This SAS program combines files of the same type from more than one country. 

JACKGEN.SAS (and SAMPLEJACKGEN.SAS) 

This SAS macro program is used to compute weighted percentages of students within defined 
subgroups, along with their means on a specified continuous variable. This macro generates replicate 
weights and computes standard errors using the jackknife repeated replication (JRR) methodology. 
The analysis variable can be any continuous variable. When computing average achievement scores 
with plausible values, the macro JACKPV.SAS should be used. 

JACKPV.SAS (and SAMPLEJACKGPV.SAS) 

This SAS macro program is used to compute weighted percentages of students within defined 
subgroups, along with their average achievement on an achievement scale using the available 
plausible values. This macro generates replicate weights and computes standard errors using the 
jackknife repeated replication (JRR) and multiple imputation methodologies. This macro should be 
used when achievement plausible values are used in an analysis. 

JACKREG.SAS (and SAMPLEJACKREG.SAS) 

This SAS macro program is used to compute regression coefficients and their standard errors within 
defined subgroups. This macro can be used with any analysis variable, but is not appropriate for 
analyzing achievement with plausible values. 

JACKREGP.SAS (and SAMPLEJACKREGP.SAS) 

This SAS macro program is used to compute regression coefficients and their standard errors within 
defined subgroups when using achievement plausible values as the dependent variable. 
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EXAMPLE1.SAS, EXAMPLE2.SAS, EXAMPLE3.SAS, 

EXAMPLE4.SAS, EXAMPLE5.SAS, EXAMPLE6.SAS 

These are the programs used in the example analyses presented in this chapter. 

The four SAS macros— JACKGEN, JACKPV, JACKREG, and JACKREGP— have a 
corresponding sample program that calls the respective macro and prints out the results. These 
sample programs are discussed later in this chapter (See 3.6 SAS Macros to Compute Statistics and 
their Standard Errors). 

3.3 Converting the SAS Export Files 

The PIRLS 2011 International Database provides a program called CONVERT. SAS that converts 
the SAS export files provided in the International Database into SAS data files. This conversion is 
necessary because all of the SAS macros and SAS programs presented in this chapter require the use 
of SAS data files. To convert SAS export files into SAS data files, users should apply the following 
steps: 

1 . Open the SAS program file CONVERT. SAS. 

2. At the beginning of the program, specify the study in the parameter “STUDY” — either “R3” for 
PIRLS, or “LI” for prePIRLS. 

3. Specify the data file type in the parameter “TYPE.” 

4. Specify the path where the SAS export files are located in the parameter “EXPPATH.” 

5. Specify the folder where the converted SAS data files will be located in the parameter 
“DATPATH.” 

6. List all of the countries of interest in the parameter “COUNTRY.” By default, all PIRLS 2011 
countries are listed and the program will automatically select the appropriate list based on the 
study specified in step 2. 

7. Submit the edited code for processing by SAS. 

An example of the CONVERT program is presented in Exhibit 3.1. This example converts the 
SAS export files of type ASG for all PIRLS 2011 countries. For this example, all SAS export files 
are located in the folder titled “C:\PIRLS201 l\Data\SAS_Data,” where the converted SAS data files 
also will be located. 

Users are advised to run the CONVERT program for all countries and all file types. The file 
types are ACG, ASA, ASG, ASH, ASR, AST, and ATG. These file types are described in Chapter 4. 
In principle, this program needs to be run only once for each file type and should be one of the first 
thing users do with the PIRLS 2011 International Database before undertaking any data analyses, 
particularly the data analysis examples in this User Guide. 
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Exhibit 3.1 : Example of CONVERT Program 


%LET STUDY = R3 ; 

%LET TYPE = ASG ; 

%LET EXP PATH = C:\PIRLS201l\Data\SAS_Data\ ; 

%LET DATPATH = C:\PIRLS201l\Data\SAS_Data\ ; 

% MACRO DOIT ; 

%LET COUNTRY = < List of PIRLS 2011 countries > ; 

%LET 1=1; 

%DO %WHILE (%LENGTH (%SCAN (&COUNTRY, &I) ) ) ; 

%LET CTRY = %SCAN (&COUNTRY, &i) ; 

PROC CIMPORT FILE = " &EXPPATH&TYPE&CTRY&STUDY . . EXP " 
DATA = " &DATPATH&TYPE&CTRY&STUDY " ; 

%LET I = %EVAL ( &I +1) ; 

%END ; 

%MEND DOIT ; 

%DOIT ; 


3.4 Scores for the PIRLS/prePIRLS 201 1 Items 

Student achievement in PIRLS and prePIRLS 2011 is represented by sets of five plausible values for 
overall reading, its two purposes for reading, and its two processes of comprehension, and these are 
the preferred scores for any analysis of student achievement. However, analyzing performance on 
individual items may be of interest to some users. Carrying out such analyses may require that the 
individual items in the PIRLS 2011 International Database be assigned their correctness score levels, 
rather than the actual response options selected by students for multiple choice items or the codes 
given to students’ responses to constructed response items. The International Database provides SAS 
programs to perform this task. 

For multiple choice items, numbers 1 through 4 are used to represent response options A through 
D, respectively, in the PIRLS and prePIRLS 2011 achievement data files. These responses must be 
converted to their appropriate score level (“1” for correct and “0” for incorrect) based on each 
multiple choice item’s correct response key. For constructed response items, worth a total of one, 
two, or three points, one-digit codes are used to represent the students’ written responses in the 
achievement data files. These codes actually represent the point value of the students’ response and 
thus do not require recoding. 

For both types of items, special codes are set aside to represent missing data as either “Not 
Administered,” “Omitted,” or “Not Reached.” These special missing codes also must be recoded in 
order to carry out specific item-level analyses. By default, the “Not Administered” response code is 
left as missing and the “Omitted” and “Not Reached” response codes are recoded as incorrect. These 
default settings can be modified within the programs, depending on the requirements of the item- 
level analyses. For example, “Not Reached” responses were treated as missing for the purpose of 
calibrating the PIRLS and prePIRLS 201 1 items, whereas they were treated as incorrect when 
deriving achievement scores for students. 
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The PIRLS 2011 International Database includes two SAS programs — ASASCRR3.SAS for the 
PIRLS items, and ASASCRL1.SAS for the prePIRLS items — which will recode the responses to 
individual items from the achievement data files to their appropriate score levels. To score each 
individual PIRLS and prePIRLS 201 1 item, the program code in the ASASCRR3 and ASASCRL1 
programs must be adapted by doing the following steps: 

1. Open the SAS program file ASASCRR3.SAS or ASASCRL1.SAS. 

2. Specify the folder where the SAS data files are located in the “LIBNAME” statement. 

3. List all of the countries of interest in the parameter “COUNTRY” (by default, all PIRLS or 
prePIRLS 2011 countries are listed). 

4. Submit the edited code for processing by SAS. 

Each program uses the student achievement data files as input (ASA), recodes the individual 
items, and saves the results in SAS data files that have “ASC” instead of “ASA” as the first three 
characters in their file names. Exhibit 3.2 shows a condensed version of the ASASCRR3 program to 
score the individual PIRLS 2011 items. 

Exhibit 3.2: Example of ASASCRM5/BSASCRM5 Programs 


LIBNAME LIBDAT "C:\PIRLS201l\Data\SAS_Data\" ; 

%LET COUNTRY = < List of PIRLS 2011 countries > ; 

%LET ARIGHT = < List of multiple-choice items where A is correct > ; 

%LET BRIGHT = < List of multiple-choice items where B is correct > ; 

%LET CRIGHT = < List of multiple-choice items where C is correct > ; 

%LET DRIGHT = < List of multiple-choice items where D is correct > ; 

%LET CONSTR = < List of constructed-response items > ; 

% MACRO SCORE IT (ITEM, TYPE, RIGHT, NR, NA, OM, OTHER) ; 


%MEND SCORE IT ; 
% MACRO DOIT ; 


DO 

OVER 

ARIGHT ; 

■ %SCOREIT 

(ARIGHT, 

"MC" , 

1, 

• R, 

• A, . , 

.1) 

; END 

DO 

OVER 

BRIGHT ; 

• %SCOREIT 

(BRIGHT, 

"MC" , 

2 , 

.R, 

• A, . , 

.1) 

; END 

DO 

OVER 

CRIGHT ; 

■ %SCOREIT 

(CRIGHT, 

"MC" , 

3, 

.R, 

• A, . , 

.1) 

; END 

DO 

OVER 

DRIGHT ; 

■ %SCOREIT 

(DRIGHT, 

"MC" , 

4, 

.R, 

• A, . , 

.1) 

; END 

DO 

OVER 

CONSTR ; 

• %SCOREIT 

(CONSTR, 

"CR" , 

, 

.R, 

• A, . , 

.1) 

; END 


%MEND DOIT ; 
%DOIT ; 


If “Not Reached” responses are to be treated as missing rather than incorrect, users should 
replace the following statement (which appears twice in the programs): 


IF &ITEM = &NR THEN SCORE = 0 ; 
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with this statement: 


IF &ITEM = &NR THEN SCORE = . ; 


3.5 Joining the PIRLS/prePIRLS 201 1 Data Files 

The PIRLS 201 1 International Database contains separate data files for each country. The 
International Database provides a SAS program called JOIN. S AS that joins individual country data 
files of a particular type into a single aggregated data file, facilitating joint analyses involving more 
than one country. The JOIN program, however, can only join SAS data files of the same type, and 
thus can be used for the following data file types: ACG, ASA, ASC, ASG, ASH, ASR, AST, and 
ATG. To create a SAS data file with more than one country’s data, users should do the following: 

1 . Open the SAS program file JOIN. SAS. 

2. At the beginning of the program, specify the study in the parameter “STUDY” — either “R3” for 
PIRLS, or “LI” for prePIRLS. 

3. Specify the data file type in the parameter “TYPE.” 

4. Specify the folder where the SAS data files are located in the LIBDAT statement. 

5. List all of the countries of interest in the parameter “COUNTRY” (by default, all PIRLS and 
prePIRLS 2011 countries are listed). 

6. Submit the edited code for processing by SAS. 

An example of the JOIN program is displayed in Exhibit 3.3. It joins the PIRLS 2011 student 
background data files (ASG) of all countries. All country data files are located in the folder titled 
“C:\PIRLS201 l\Data\SAS_Data” and the resulting data file ASGALLR3 will be saved in this folder. 

Exhibit 3.3: Example of JOIN Program 


%LET STUDY = R3 ; 

%LET TYPE = ASG ; 

LIBNAME LIBDAT "C:\PIRLS201l\Data\SAS_Data\" ; 

% MACRO DOIT ; 

%LET COUNTRY = < List of PIRLS 2011 countries > ; 

DATA &TYPE . ALL&STUDY ; 

SET %LET 1=1; 

%DO %WHILE (%LENGTH (%SCAN (&COUNTRY, &I) ) ) ; 

%LET CTRY = %SCAN (&COUNTRY, &I) ; 

LIBDAT . &TYPE&CTRY&STUDY 
%LET I = %EVAL ( &I +1) ; 

%END ; ; 

PROC SORT DATA = &TYPE . ALL&STUDY OUT = LIBDAT . &TYPE . ALL&STUDY ; 
BY &SORTVARS ; 

%MEND DOIT ; 

%DOIT ; 
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3.6 SAS Macros to Compute Statistics and their Standard Errors 


This section describes the four SAS macros— JACKGEN, JACKPV, JACKREG, and JACKREGP — 
needed to compute specific statistics with their correct standard errors, along with sample SAS 
programs to demonstrate their use. Users are encouraged to modify the sample SAS programs and 
familiarize themselves with their functioning. The four SAS macros, however, do not require any 
modifications. 

Each SAS macro serves a specific analytical purpose. These macros ensure that analyses of the 
PIRLS and prePIRLS 201 1 data are done properly. Sampling weights are used and standard errors 
are computed using the jackknife repeated replication (JRR) method. Furthermore, achievement 
scores are based on sets of five plausible values that take into account the measurement error arising 
from the test design and the IRT scaling methodology. The macros that make use of these plausible 
values effectively perform five analyses — one for each plausible value — and aggregate the results to 
produce accurate estimates of achievement and standard errors that incorporate both sampling and 
imputation errors. 

The sample SAS programs presented in this section all use as input the SAS data file 
ASGALLR3, which contains the PIRLS 2011 student background data files of all participating 
countries. In all sample programs, <datpath> must be edited to specify the folder where the 
ASGALLR3 file is located, as well as <macpath> to indicate the folder where the SAS macros are 
located. 

Computing Means and their Standard Errors — JACKGEN 

The JACKGEN macro is used to compute percentages and means of continuous variables with their 
standard errors. We will demonstrate its use with a sample SAS program that calls the macro 
JACKGEN to compute the percentages of students within specified subgroups and their mean on a 
variable of choice. The macro also computes the appropriate standard errors for the percentages and 
means. However, this macro is not appropriate for analyzing achievement scores based on plausible 
values; the JACKPV macro should be used for this purpose. 

The JACKGEN macro is a self-contained program, located in the program file JACKGEN.SAS, 
and should not be modified. It essentially computes sets of replicate weights using the sampling and 
weighting variables, aggregates the data by subgroups using the replicate weights, and then computes 
and stores the desired statistics in a SAS working file called FINAL. 

The macro JACKGEN is included in a SAS program by issuing the following command: 


% INCLUDE "<macpath> JACKGEN. SAS" ; 


where <macpath> points to the folder in which the SAS macro JACKGEN.SAS is located. This 
macro requires that the following several parameters be specified as input when it is invoked: 

WGT The sampling weight to be used in the analysis. Generally, TOTWGT should be used. 
TCHWGT should be used when analyzing teacher data. 
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JKZ The variable that captures the assignment of cases (typically students) to sampling 

zones. The name of this variable in the PIRLS and prePIRLS data files is JKZONE. 

JKR The variable that captures whether the case is to be dropped or have its weight 

doubled for each set of replicate weights. The name of this variable in the PIRLS and 
prePIRLS data files is JKREP. 

NJKZ The number of replicate weights to be generated when computing the standard errors. 

The value of NJKZ should be set to 75, the maximum possible value across all 
participating countries. 

CVAR The list of variables that are to be used to define the subgroups. The list can consist of 

one or more variables. We recommend that users always include IDCNTRY as the 
first classification variable. 

DVAR The variable for which averages are to be computed. Only one variable can be listed 
and it should be a continuous variable. Plausible values of achievement scores should 
not be specified here. 

INFILE The name of the SAS data file that contains the data being analyzed. It is important to 
emphasize that this SAS data file must include only those cases that are of interest in 
the analysis. If users want to have specific cases excluded from the analysis 
(e.g., students with missing data), this should be done prior to invoking the macro. 

The JACKGEN macro is invoked by a SAS program using the conventional SAS notation for 
invoking macros. This involves listing the macro name followed by the list of parameters in 
parenthesis, each separated by a comma. For example, the JACKGEN macro invoked using the 
following statement: 


%JACKGEN (TOTWGT, JKZONE, JKREP, 75, IDCNTRY ITSEX, ASDAGE, ASGALLR3 ) ; 


will compute the average age (ASDAGE) of PIRLS 2011 students by gender (ITSEX) and their 
standard errors within each country (IDCNTRY), using the weighting variable TOTWGT. It also 
will compute the percentages of girls and boys and their standard errors within each country. The 
data will be read from the data file ASGALLR3 and the standard errors will be computed based on 
75 sets of replicate weights. 

The results of the JACKGEN macro are stored in a SAS working file called FINAL, which is 
stored in the default folder used by SAS. The following variables are contained in this results file: 

Classification Variables 

All classification variables are kept in the results file. In the example invocation above, there are two 
classification variables: IDCNTRY and ITSEX. There is one record in the results file for each 
subgroup defined by the categories of the classification variables. 


PIRLS 201 1 USER GUIDE FOR THE INTERNATIONAL DATABASE 

46 CHAPTER 3 



TIMSS & PIRLS 

International Study Center 

Lynch School of Education, Boston College 


N 


This variable contains the number of valid cases for each subgroup defined by the classification 
variables. In the example, it is the number of girls and boys with valid data in each country’s sample. 

Weight Variable 

The weight variable contains the sum of weights within each subgroup defined by the classification 
variables. In the example, this variable is TOTWGT, because TOTWGT was specified as the 
weighting variable. This variable will be an estimate of the total population within each subgroup. 

MNX 

This variable contains the estimated averages of the specified analysis variable by subgroup. 

MNX_SE 

This variable contains the standard errors of the estimated averages by subgroup. 

PCT 

This variable contains the estimated percentages of students in each subgroup for the last 
classification variable listed. In the example, it is the percentage of girls and boys within each 
country. 

PCT_SE 

This variable contains the standard errors of the estimated percentages. 

The contents of the FINAL file can be printed using the SAS PRINT procedure. The sample SAS 
program that invokes the JACKGEN macro is presented in Exhibit 3.4, along with a printout of the 
results. This program is available in the International Database as the file called 
SAMPLE JACKGEN. SAS. It produces the average age of fourth grade girls and boys in all countries, 
although the exhibit shows the results only for the first four countries. 

As shown in the first two lines of the results in Exhibit 3.4, there were 3,048 girls in Australia’s 
PIRLS sample, representing 123,760 girls in the entire population. The average age of fourth grade 
girls in Australia was estimated to be 9.96, with a standard error of 0.01. Girls constituted 
49.26 percent of Australia’s fourth grade student population (standard error of 1.06). Conversely, 
Australia sampled 3,078 boys, representing 127,453 boys in the entire population. The estimated 
average age of fourth grade boys in Australia was 10.03, with a standard error of 0.01. Boys 
constituted 50.74 percent of Australia’s fourth grade student population (standard error of 1.06). 
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Exhibit 3.4: Sample SAS Program Invoking the SAS Macro JACKGEN with Results 


%LET STUDY = R3 ; 

LIBNAME Pll " <datpath> " ; 

% INCLUDE "<macpath> JACKGEN. SAS" ; 

DATA ASGALL&STUDY ; 

SET Pll . ASGALL&STUDY ; 

WHERE NMISS (ITSEX, ASDAGE) = 0 ; 

PROC FORMAT LIBRARY = WORK ; 

VALUE COUNTRY 

< list PIRLS 2011 country formats > ; 

VALUE SEX 

1 = "GIRL" 

2 = "BOY" ; 

% JACKGEN (TOTWGT , JKZONE , JKREP , 75, IDCNTRY ITSEX, ASDAGE, ASGALL&STUDY) ; 

PROC PRINT DATA = FINAL NOOBS ; 

VAR IDCNTRY ITSEX N TOTWGT MNX MNX_SE PCT PCT_SE ; 

FORMAT IDCNTRY COUNTRY. ITSEX SEX. N 6.0 TOTWGT 10.0 
MNX MNX_SE PCT PCT_SE 6.2 ; 


IDCNTRY 

ITSEX 

N 

TOTWGT 

MNX 

MNX_SE 

PCT 

PCT_SE 

AUSTRALIA 

GIRL 

3048 

123760 

9 . 96 

0 . 01 

49.26 

1 . 06 

AUSTRALIA 

BOY 

3078 

127453 

10 . 03 

0 . 01 

50 . 74 

1 . 06 

AUSTRIA 

GIRL 

2274 

38141 

10.21 

0 . 01 

48.69 

1 . 18 

AUSTRIA 

BOY 

2396 

40192 

10.30 

0 . 01 

51.31 

1 . 18 

AZERBAIJAN 

GIRL 

2241 

49037 

10.24 

0 . 03 

47 . 14 

0 . 85 

AZERBAIJAN 

BOY 

2640 

54977 

10 . 19 

0 . 02 

52 . 86 

0 . 85 

BELGIUM (FRENCH) 

GIRL 

1814 

24380 

10 . 05 

0 . 02 

48 . 91 

0 . 93 

BELGIUM (FRENCH) 

BOY 

1909 

25464 

10 . 12 

0 . 03 

51 . 09 

0 . 93 


Computing Achievement Means and their Standard Errors — JACKPV 

The JACKPV macro computes percentages and average achievement scores using plausible values. 

It makes use of the sampling weights, the jackknifing algorithm to compute sampling variances, and 
the five plausible values to compute imputation variances. It effectively performs five analyses — one 
for each plausible value — and aggregates the results to produce accurate estimates of average 
achievement and standard errors that incorporate both sampling and imputation errors. 

A second sample program demonstrates the use of the JACKPV macro, which computes the 
percentages of students within specified subgroups and their average achievement scores. This SAS 
macro also computes the appropriate standard errors for those percentages and achievement 
averages. 

The JACKPV macro is a self-contained program, located in the program file JACKPV.SAS, and 
should not be modified. It essentially computes sets of replicate weights using the sampling and 
weighting variables, aggregates the data by subgroups using the replicate weights, and then computes 
and stores the desired statistics in a SAS working file called FINAL. The macro aggregates data 
across all plausible values to obtain the correct results. 

The SAS macro JACKPV is included in a SAS program by issuing the following command: 


% INCLUDE "<macpath> JACKPV. SAS" ; 
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where <macpath> points to the folder in which the SAS macro program JACKPV.SAS is located. 
This macro requires that the following several parameters be specified as input when it is invoked: 

WGT The sampling weight to be used in the analysis. Generally, TOTWGT should be used. 
TCHWGT should be used when analyzing teacher data. 

JKZ The variable that captures the assignment of cases to sampling zones. The name of 

this variable in the PIRLS and prePIRLS data files is JKZONE. 

JKR The variable that captures whether the case is to be dropped or have its weight 

doubled for each set of replicate weights. The name of this variable in the PIRLS and 
prePIRLS data files is JKREP. 

NJKZ The number of replicate weights to be generated when computing the standard errors. 

The value of NJKZ should be set to 75, the maximum possible value across all 
participating countries. 

CVAR The list of variables that are to be used to define the subgroups. The list can consist of 

one or more variables. We recommend that users always include IDCNTRY as the 
first classification variable. 

ROOTPV The variable root used to identify the set of plausible values for the achievement score 
of interest. It corresponds to the first seven characters of the plausible values variable 
name. For example, the root of the fourth grade overall reading plausible values is 
ASRREAO. 

NPV The number of plausible values that will be used for the analysis. Generally, it is set to 

five in order to use all five available plausible values for analysis. 

INFILE The name of the SAS data file that contains the data being analyzed. It is important to 
emphasize that this SAS data file must include only those cases that are of interest in 
the analysis. If users want to have specific cases excluded from the analysis 
(e.g., students with missing data), this should be done prior to invoking the macro. 

The JACKPV macro is invoked by a SAS program using the conventional SAS notation for 
invoking macros. This involves listing the macro name followed by the list of parameters in 
parenthesis, each separated by a comma. For example, the JACKPV macro invoked using the 
following statement: 


% JACKPV (TOTWGT, JKZONE, JKREP, 75, IDCNTRY ITSEX, ASRREAO, 5, ASGALLR3 ) ; 


will compute the average reading achievement (ASRREAO 1 through ASRREA05) by gender 
(ITSEX) within each country (IDCNTRY) and their standard errors, using the weighting variable 
TOTWGT. The macro uses all five plausible values to compute these statistics. It also will compute 
the percentages of girls and boys within each country, along with their standard errors. The data will 
be read from the data file ASGALLR3 and the standard errors will be computed based on 75 sets of 
replicate weights. 
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The results of the JACKPV macro are stored in a SAS working file called FINAL, which is 
stored in the default folder used by SAS. The following variables are contained in this results file: 

Classification Variables 

All classification variables are kept in the results file. In this example, there are two classification 
variables: IDCNTRY and ITSEX. There is one record in the results file for each subgroup defined by 
the categories of the classification variables. 

N 

This variable contains the number of valid cases for each subgroup defined by the classification 
variables. In the example, it is the number of girls and boys with valid data in each country’s sample. 

Weight Variable 

The weight variable contains the sum of weights within each subgroup defined by the classification 
variables. In the example, this variable is called TOTWGT, because TOTWGT was specified as the 
weighting variable. This variable will be an estimate of the total population within each subgroup. 

MNPV 

This variable contains the estimated average achievement by subgroup, based on the plausible 
values. 

MNPV_SE 

This variable contains the standard errors of the estimated average achievement by subgroup, based 
on the plausible values. 

PCT 

This variable contains the estimated percentages of students in each subgroup for the last 
classification variable listed. In the example, it is the percentage of girls and boys within each 
country. 

PCT_SE 

This variable contains the standard errors of the estimated percentages. 

The contents of the FINAL file can be printed using the SAS PRINT procedure. The sample SAS 
program that invokes the JACKPV macro is shown in Exhibit 3.5, along with a printout of the 
results. This program is available in the International Database as the file called 
SAMPLE JACKPV. SAS. It produces the average reading achievement for girls and boys in all 
countries, although Exhibit 3.5 gives the results only for the first four countries. 
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Exhibit 3.5: Sample SAS Program Invoking the SAS Macro JACKPV with Results 


%LET STUDY = R3 ; 

LIBNAME Pll " <datpath> " ; 

% INCLUDE "<macpath> JACKPV. SAS" ; 

DATA ASGALL&STUDY ; 

SET Pll . ASGALL&STUDY ; 

WHERE NMISS (ITSEX) = 0 ; 

PROC FORMAT LIBRARY = WORK ; 

VALUE COUNTRY 

< list PIRLS 2011 country formats > ; 

VALUE SEX 

1 = "GIRL" 

2 = "BOY" ; 

% JACKPV (TOTWGT , JKZONE , JKREP, 75, IDCNTRY ITSEX, ASRREAO , 5, ASGALL&STUDY) ; 

PROC PRINT DATA = FINAL NOOBS ; 

VAR IDCNTRY ITSEX N TOTWGT MNPV MNPV_SE PCT PCT_SE ; 

FORMAT IDCNTRY COUNTRY. ITSEX SEX. N 6.0 TOTWGT 10.0 
MNPV MNPV_SE PCT PCT_SE 6.2 ; 


IDCNTRY 

ITSEX 

N 

TOTWGT 

MNPV 

MNPV_SE 

PCT 

PCT_SE 

AUSTRALIA 

GIRL 

3048 

123760 

535 . 79 

2 .67 

49.26 

1 . 06 

AUSTRALIA 

BOY 

3078 

127453 

519.20 

2 . 73 

50 . 74 

1 . 06 

AUSTRIA 

GIRL 

2274 

38141 

532 . 76 

2 . 18 

48 . 69 

1 . 18 

AUSTRIA 

BOY 

2396 

40192 

525 . 19 

2.32 

51.31 

1 . 18 

AZERBAIJAN 

GIRL 

2241 

49037 

469.57 

3.56 

47 . 14 

0 . 85 

AZERBAIJAN 

BOY 

2640 

54977 

455 . 82 

3.47 

52 . 86 

0 . 85 

BELGIUM (FRENCH) 

GIRL 

1815 

24388 

508 . 85 

3 . 11 

48 . 89 

0 . 93 

BELGIUM (FRENCH) 

BOY 

1912 

25491 

503.51 

3 . 11 

51 . 11 

0 . 93 


As shown in the first two lines of the results in Exhibit 3.5, the average reading achievement of 
fourth grade girls in Australia was estimated at 535.79, with a standard error of 2.67. The average 
reading achievement of fourth grade boys in Australia was estimated at 519.20, with a standard error 
of 2.73. 


Computing Regression Coefficients and their Standard Errors — JACKREG 

The JACKREG macro performs a multiple linear regression between a dependent variable and a set 
of independent variables. A third sample program demonstrates the use of the JACKREG macro, 
which computes the regression coefficients and their standard errors. This macro is not appropriate 
for regression analyses using achievement scores as the dependent variable. The JACKREGP macro 
should be used for this purpose. 

The JACKREG macro is a self-contained program, located in the program file JACKREG. SAS, 
and should not be modified. It computes sets of replicate weights using the sampling and weighting 
variables, performs a linear regression by subgroup using the replicate weights, and then computes 
and stores the desired statistics in a SAS working file called REG. 

The SAS macro JACKREG is included in a SAS program by issuing the following command: 


% INCLUDE "<macpath> JACKREG. SAS" ; 
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where <macpath> points to the specific folder in which the SAS macro program JACKREG.SAS is 
located. This macro requires that the following several parameters be specified as input when it is 
invoked: 

WGT The sampling weight to be used in the analysis. Generally, TOTWGT should be used. 
TCHWGT should be used when analyzing teacher data. 

JKZ The variable that captures the assignment of cases to sampling zones. The name of 

this variable in the PIRLS and prePIRLS data files is JKZONE. 

JKR The variable that captures whether the case is to be dropped or have its weight 

doubled for each set of replicate weights. The name of this variable in the PIRLS and 
prePIRLS data files is JKREP. 

NJKZ The number of replicate weights to be generated when computing the standard errors. 
The value of NJKZ should be set to 75, the maximum possible value across all 
participating countries. 

CVAR The list of variables that are to be used to define the subgroups. The list can consist of 
one or more variables. We recommend that users always include IDCNTRY as the 
first classification variable. 

XVAR The list of independent variables used as predictors in the regression model. The 
independent variables can be either continuous or categorical (e.g., ITSEX). 

DVAR The dependent variable to be predicted by the list of independent variables specified 
in XVAR. Only one variable can be listed, and plausible values of achievement scores 
should not be specified here. 

INFILE The name of the SAS data file that contains the data being analyzed. It is important to 
emphasize that this SAS data file must include only those cases that are of interest in 
the analysis. If users want to have specific cases excluded from the analysis, for 
example students with missing data, this should be done prior to invoking the macro. 

The JACKREG macro is invoked by a SAS program using the conventional SAS notation for 
invoking macros. This involves listing the macro name followed by the list of parameters in 
parenthesis, each separated by a comma. For example, the JACKREG macro invoked using the 
following statement: 


%JACKREG (TOTWGT, JKZONE, JKREP, 75, IDCNTRY, REGSEX, ASBGSLR, ASGALLR3 ) ; 


will perform a linear regression with gender (REGSEX) as a predictor of the students’ score on the 
Students Like Reading contextual scale (ASBGSLR), using the weighting variable TOTWGT. 2 It 
will compute the regression coefficients and their standard errors. The data will be read from the data 
file ASGALLR3 and the standard errors will be computed based on 75 replicate weights. 


2 The Students Like Reading contextual scale is described in Methods and Procedures in TIMSS and PIRLS 2011 (Martin & Mullis, 2012). 
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The results of the JACKREG macro are stored in a SAS working file called REG, which is stored 
in the default folder used by SAS. The following variables are contained in this results file: 

Classification Variables 

All classification variables are kept in the results file. In this example, there is a single classification 
variable IDCNTRY. There is one record in the results file for each subgroup defined by the 
categories of the classification variables. 

N 

This variable contains the number of valid cases for each subgroup defined by the classification 
variables. In the example, it is the number of students with valid data in each country’s sample. 

MULT_RSQ 

The squared multiple correlation coefficient (R 2 ) for the regression model applied in each subgroup. 
SS_RES, SS_REG, SS_TOTAL 

The residual, regression, and total weighted sums of squares for the regression model applied in each 
subgroup. 

Regression Coefficients and Standard Errors (B## and B##.SE) 

The regression coefficients for the intercept and the predictor variables with their respective standard 
errors. The regression coefficients are numbered sequentially, starting with zero (BOO) for the 
intercept, and based on the order of the predictor variables as specified in the parameter XV AR. 

The contents of the REG file can be printed using the SAS PRINT procedure. The sample SAS 
program that invokes the JACKREG macro is displayed in Exhibit 3.6, along with a printout of the 
results. This program is available in the International Database as the file called 
SAMPLE JACKREG. SAS. It performs a linear regression in each country, with the variable 
REGSEX as a predictor of the fourth grade students’ score on the Students Like Reading contextual 
scale (ASBGSLR). The exhibit displays the results for the first four countries. 

The regression performed by the sample program uses the independent variable REGSEX, which 
is a “dummy-coded” version of ITSEX, such that the value “0” represents the girls and the value “1” 
represents the boys. By performing this recoding, the intercept BOO will be the estimated average 
score of fourth grade girls, whereas the regression coefficient B01 will be the estimated increase in 
average score for boys. This will allow us to determine if the difference in average score between 
girls and boys is statistically significant. 

As shown in the first line of the results in Exhibit 3.6, the estimated average score of girls in 
Australia on the ASBGSLR scale (BOO) was 10.34, with a standard error of 0.06. The boys in 
Australia were an estimated 0.82 points lower (B01) than the girls on that scale (the sign of this 
coefficient being negative), with an estimated standard error of 0.07. Note that this difference is 
statistically significant at the 95% confidence level. 
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Exhibit 3.6: Sample SAS Program Invoking the SAS Macro JACKREG with Results 


%LET STUDY = R3 ; 

LIBNAME Pll " <datpath> " ; 

% INCLUDE "<macpath> JACKREG. SAS" ; 

DATA ASGALL&STUDY ; 

SET Pll . ASGALL&STUDY ; 

WHERE NMISS (ITSEX, ASBGSLR) = 0 ; 

SELECT (ITSEX) ; 

WHEN (1) REGSEX = 0 ; * GIRLS ; 

WHEN (2) REGSEX = 1 ; * BOYS ; 

OTHERWISE REGSEX = . ; 

END ; 

PROC FORMAT LIBRARY = WORK ; 

VALUE COUNTRY 

< list PIRLS 2011 country formats > ; 

% JACKREG (TOTWGT , JKZONE , JKREP , 75, IDCNTRY, REGSEX, ASBGSLR, ASGALL&STUDY) ; 
PROC PRINT DATA = REG NOOBS ; 

VAR IDCNTRY N MULT_RSQ SS_TOTAL SS_REG BOO B00_SE B01 B01_SE ; 

FORMAT IDCNTRY COUNTRY. N 6.0 MULT_RSQ 5.3 SS_TOTAL SS_REG 10.0 
BOO BO 0_SE BO 1 B01_SE 6.2 ; 


IDCNTRY 

N 

MULT_RSQ 

SS_TOTAL 

SS_REG 

BOO 

B0 0_SE 

B0 1 

B01_SE 

AUSTRALIA 

6022 

0 . 038 

1092334 

41737 

10.34 

0 . 06 

-0 . 82 

0 . 07 

AUSTRIA 

4657 

0 . 059 

365610 

21536 

10.51 

0 . 06 

-1 . 05 

0 . 08 

AZERBAIJAN 

4533 

0 . 002 

246599 

522 

10.45 

0 . 06 

-0 . 15 

0 . 06 

BELGIUM (FRENCH) 

3709 

0 . 022 

202774 

4432 

10.41 

0 . 06 

-0 . 60 

0 . 08 


Computing Regression Coefficients and their Standard Errors with Achievement — JACKREGP 

The JACKREGP macro is used to perform a multiple linear regression between a set of plausible 
values as the dependent variable and a set of independent variables. It computes the regression 
coefficients and their standard errors, making use of the sampling weights, the jackknifing algorithm 
to compute sampling variances, and the five plausible values to compute imputation variances. It 
effectively performs five regression analyses — one for each plausible value — and aggregates the 
results to produce accurate estimates of the regression coefficients and standard errors that 
incorporate both sampling and imputation errors. Below, we present a fourth sample program to 
demonstrate the use of the JACKREGP macro. 

The JACKREGP macro is a self-contained program, located in the program file 
JACKREGP. SAS, and should not be modified. It computes sets of replicate weights using the 
sampling and weighting variables, performs a multiple linear regression by subgroups using the 
replicate weights, and then computes and stores the desired statistics in a SAS working file called 
REG. 

The SAS macro JACKREGP is included in a SAS program by issuing the following command: 


% INCLUDE " <macpath> JACKREGP . SAS " ; 
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where <macpath> points to the specific folder in which the SAS macro program JACKREGP.SAS is 
located. This macro requires that the following several parameters be specified as input when it is 
invoked: 

WGT The sampling weight to be used in the analysis. Generally, TOTWGT should be used. 
TCHWGT should be used when analyzing teacher data. 

JKZ The variable that captures the assignment of cases to sampling zones. The name of 

this variable in the PIRLS and prePIRLS data files is JKZONE. 

JKR The variable that captures whether the case is to be dropped or have its weight 

doubled for each set of replicate weights. The name of this variable in the PIRLS and 
prePIRLS data files is JKREP. 

NJKZ The number of replicate weights to be generated when computing the standard errors. 
The value of NJKZ should be set to 75, the maximum possible value across all 
participating countries. 

CVAR The list of variables that are to be used to define the subgroups. The list can consist of 
one or more variables. We recommend that users always include IDCNTRY as the 
first classification variable. 

XVAR The list of independent variables used as predictors in the regression model. The 
independent variables can be either continuous or categorical, such as ITSEX for 
example. 

ROOTPV The variable root used to identify the set of plausible values for the achievement score 
of interest. It corresponds to the first seven characters of the plausible values variable 
name. Lor example, the root of the fourth grade overall reading plausible values is 
ASRREAO. 

NPV The number of plausible values that will be used for the analysis. Generally, it is set to 

five to use all five available plausible values for analysis. 

INL ILE The name of the SAS data file that contains the data being analyzed, this number is 
important to emphasize that this SAS data file must include only those cases that are 
of interest in the analysis. If users want to have specific cases excluded from the 
analysis, for example students with missing data, this should be done prior to 
invoking the macro. 

The JACKREGP macro is invoked by a SAS program using the conventional SAS notation for 
invoking macros. This involves listing the macro name followed by the list of parameters in 
parenthesis, each separated by a comma, for example, the JACKREGP macro invoked using the 
following statement: 


% JACKREGP (TOTWGT, JKZONE, JKREP, 75, IDCNTRY, REGSEX, ASRREAO, 5, ASGALLR3 ) ; 
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will perform a linear regression with gender (REGSEX) as a predictor of reading achievement based 
on its five plausible values (ASRREA01 through ASRREA05), using the weighting variable 
TOTWGT. It will compute the regression coefficients and their standard errors. The data will be read 
from the data file ASGALLR3 and the standard errors will be computed based on 75 replicate 
weights. 

The results of the JACKREGP macro are stored in a SAS working file called REG, which is 
stored in the default folder used by SAS. The following variables are contained in this results file: 

Classification Variables 

All classification variables are kept in the results file. In this example, there is a single classification 
variable IDCNTRY. There is one record in the results file for each subgroup defined by the 
categories of the classification variables. 

N 

This variable contains the number of valid cases for each subgroup defined by the classification 
variables. In the example, it is the number of students with valid data in each country’s sample. 

MULT_RSQ 

The squared multiple correlation coefficient (R 2 ) for the regression model applied in each subgroup. 
SS_RES, SS_REG, SS_TOTAL 

The residual, regression, and total weighted sums of squares for the regression model applied in each 
subgroup. 

Regression Coefficients and Standard Errors (B## and B##.SE) 

The regression coefficients for the predictor variables and the intercept with their respective standard 
errors. The regression coefficients are numbered sequentially, starting with zero (BOO) for the 
intercept, and based on the order of the predictor variables as specified in the parameter XV AR. 

The contents of the REG file can be printed using the SAS PRINT procedure. The sample SAS 
program invoking the JACKREGP macro is presented in Exhibit 3.7, along with a printout of the 
results. This program is available in the International Database as the file called 
SAMPLEJACKREGP.SAS. It performs a linear regression in each country, with the variable 
REGSEX as a predictor of fourth grade reading achievement. The exhibit displays the results for the 
first four countries. 

The regression performed by our sample program uses the variable REGSEX that was defined in 
our previous example. By using REGSEX, the intercept BOO will be the estimated average reading 
achievement of fourth grade girls, whereas the regression coefficient B01 will be the estimated 
difference in the average reading achievement of boys. This will allow us to determine if fourth 
grade reading achievement is significantly different between girls and boys. 
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Exhibit 3.7: Sample SAS Program Invoking the SAS Macro JACKREGP with Results 


%LET STUDY = R3 ; 

LIBNAME Pll " <datpath> " ; 

% INCLUDE " <macpath> JACKREGP . SAS " ; 

DATA ASGALL&STUDY ; 

SET Pll .ASGALL&STUDY ; 

WHERE NMISS (ITSEX) = 0 ; 

SELECT (ITSEX) ; 

WHEN (1) REGSEX = 0 ; * GIRLS ; 

WHEN (2) REGSEX = 1 ; * BOYS ; 

OTHERWISE REGSEX = . ; 

END ; 

PROC FORMAT LIBRARY = WORK ; 

VALUE COUNTRY 

< list PIRLS 2011 country formats > ; 

% JACKREGP (TOTWGT , JKZONE , JKREP , 75, IDCNTRY, REGSEX, ASRREAO , 5, ASGALL&STUDY) ; 
PROC PRINT DATA = REG NOOBS ; 

VAR IDCNTRY N MULT_RSQ SS_TOTAL SS_REG BOO B00_SE B01 B01_SE ; 

FORMAT IDCNTRY COUNTRY. N 6.0 MULT_RSQ 5.3 SS_TOTAL SS_REG 10.0 
BOO BO 0_SE BO 1 B01_SE 6.2 ; 


IDCNTRY 

N 

MULT_RSQ 

SS_TOTAL 

SS_REG 

BOO 

B0 0_SE 

B0 1 

B01_SE 

AUSTRALIA 

6126 

0 . Oil 

1616645052 

17292861 

535 . 79 

2 .67 

-16.58 

3 . 11 

AUSTRIA 

4670 

0 . 004 

314632633 

1125018 

532 . 76 

2 . 18 

-7.58 

2.31 

AZERBAIJAN 

4881 

0 . 010 

478571881 

4903272 

469.57 

3.56 

-13 . 75 

2.34 

BELGIUM (FRENCH) 

3727 

0 . 002 

208632870 

371149 

508 . 85 

3 . 11 

-5.34 

2.34 


From the first line of the results shown in Exhibit 3.7, the estimated average reading achievement 
of fourth grade girls in Australia (BOO) was 535.79, with a standard error of 2.67. Note that these are 
the same results obtained from the JACKPV sample program (Exhibit 3.5). The fourth grade boys 
had an estimated average reading achievement 16.58 points (B01) lower than girls (the sign of this 
coefficient being negative), with an estimated standard error of 3. 1 1 . Note that this difference is 
statistically significant at the 95% confidence level. 

3.7 PIRLS/prePIRLS Analyses with Student-level Variables 

Many analyses of the PIRLS and prePIRLS 2011 data can be undertaken using only student-level 
data. Examples in the previous section illustrated the functioning of the SAS macros. This section 
presents examples of actual analyses reported in PIRLS 2011 International Results in Reading, using 
SAS programs provided in the PIRLS 2011 International Database. 

A first example computes national average achievement, whereas a second example computes 
national average achievement by gender. In both cases, the SAS macros use the sampling weights, 
implement the jackknife repeated replication method to compute appropriate sampling errors, 
effectively perform the computations five times (once for each plausible value), and aggregate the 
results to produce accurate estimates of average achievement and standard errors that incorporate 
both sampling and imputation errors. A third example expands on the second example by performing 
a test of significance on the gender difference using regression. 
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In general, to perform student-level analyses using the student background data fdes, users 
should do the following: 

1 . Identify the variables of interest in the student background data files and note any specific 
national adaptations to the variables. 

2. Retrieve the relevant variables from the student background data files, including the plausible 
values of achievement if required, classification variables, identification variables, sampling and 
weighting variables, and any other variables used in the selection of cases. 

3. Perform any necessary variable transformations or recodes. 

4. Use the macros JACKGEN and JACKREG, or JACKPV and JACKREGP if plausible values are 
involved, with the appropriate parameters. 

5. Specify the location of the data files (<datpath>) and the macros (<macpath>). 

6. Print the results file. 

Student-level Analysis with Achievement 

In our first example, we want to replicate the analysis of the overall distribution of reading 
achievement. These results are presented in Exhibit 1.1 of PIRLS 2011 International Results in 
Reading and are repeated here in Exhibit 3.8. Because the results in this exhibit are based on 
plausible values, we need to make sure that we include them when we create the file using the merge 
module, and also to indicate that our analysis will make use of achievement scores. 

We will find all of the variables we need for this analysis in the student background data files, 
including the five plausible values of fourth grade reading achievement (ASRREA01 through 
ASRREA05), the student sampling weight (TOTWGT), the variables that contain the jackknife 
replication information (JKZONE and JKREP), and the variable containing the country identification 
code (IDCNTRY). In this analysis, we will use the data for all available countries. We used the JOIN 
program, described earlier in this chapter, to join the student background data files for all countries 
into a single file called ASGALLR3. 

The SAS program used to perform this first example is presented in Exhibit 3.9 and is included in 
the International Database under the name EXAMPLE1.SAS. The results obtained from this program 
are displayed in Exhibit 3.10, although only the results of first four countries are shown for the sake 
of brevity. 

In Exhibit 3.10, the results are displayed one line per country, with the countries identified in the 
first column. The second column reports the number of valid cases, and the third the sum of weights 
of the sampled students. The next four columns report the estimated average reading achievement 
and its standard error, followed by the percentage of students in each category and its standard error. 
As shown in the first line, the average reading achievement of fourth grade students in Australia was 
527.37, with a standard error of 2.21. In this example, only IDCNTRY was used as a classification 
variable; thus, the estimated percentages are of little value because they represent the proportion of 
the TOTWGT column for each country. 
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Exhibit 3.8: Exhibit of Example Student-level Analysis with Achievement, Taken from 

PIRLS 2011 International Results in Reading (Exhibit 1 .1 ) 



100 200 300 400 500 600 700 800 


Country 


Average 
Scale Score 


Reading Achievement Distribution 


Sixth Grade Participants 


Honduras 

450 (4.8) 

® 


Morocco 

424 (3.9) 

® 


1 * Kuwait 

419 (5.2) 

® 


Botswana 

419 (4.1) 

® 



Benchmarking Participants 0 

i 3 Florida, US 569 (2.9) O 

2 Ontario, Canada 552 (2.6) O 

2 Alberta, Canada 548 (2.9) O 

Quebec, Canada 538 (2.1) O 

Andalusia, Spain 515 (2.3) O 

Dubai, UAE 476 (2.0) ® 

Maltese - Malta 457 (1.5) ® 

Abu Dhabi, UAE 424 (4.7) ® 

vEng/Afr(5)-RSA 421 (7.3) ® 

0 Republic of South Africa (RSA) tested 5th grade students receiving 
instruction in English (ENG) or Afrikaans (AFR). 



O Country average significantly higher than 
the centerpoint of the PIRLS scale 
® Country average significantly lower than 
the centerpoint of the PIRLS scale 


Percentiles of Performance 

5th 25th 75th 95th 



95% Confidence Interval for Average (±2SE) 


>K Average achievement not reliably measured because the percentage of students with achievement too low for estimation exceeds 25%. 

V Reservations about reliability of average achievement because the percentage of students with achievement too low for estimation does not exceed 25% but exceeds 1 5%. 
See Appendix C.2 for target population coverage notes 1, 2, and 3. See Appendix C.5 for sampling guidelines and sampling participation notes t and f. 

( ) Standard errors appear in parentheses. Because of rounding some results may appear inconsistent 
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SOURCE: lEA's Progress in International Reading Uteracy Study - PIRLS 201 1 





Exhibit 3.9: Example SAS Program for Student-level Analysis with Achievement (EXAMPLEI .SAS) 


LIBNAME Pll "<datpath>" ; 

% INCLUDE " <macpath> JACKPV . SAS 11 ; 

DATA ASGALLR3 ; 

SET Pll . ASGALLR3 ; 

PROC FORMAT LIBRARY = WORK ; 

VALUE COUNTRY 

< list PIRLS 2011 country formats > ; 

% JACKPV (TOTWGT , JKZONE , JKREP, 75, IDCNTRY ITSEX, ASRREAO , 5, ASGALLR3 ) ; 

PROC PRINT DATA = FINAL NOOBS ; 

VAR IDCNTRY N TOTWGT MNPV MNPV_SE PCT PCT_SE ; 

FORMAT IDCNTRY COUNTRY. N 6.0 TOTWGT 10.0 
MNPV MNPVJ3E PCT PCT_SE 6.2 ; 


Exhibit 3.1 0: Output for Example Student-level Analysis with Achievement (EXAMPLE 1 ) 


IDCNTRY 

N 

TOTWGT 

MNPV 

MNPV_SE 

PCT 

PCT_SE 

AUSTRALIA 

6126 

251213 

527.37 

2.21 

1.24 

0 . 03 

AUSTRIA 

4670 

78333 

528 . 88 

1 . 95 

0.39 

0 . 01 

AZERBAIJAN 

4881 

104014 

462.30 

3.33 

0.51 

0 . 02 

BELGIUM (FRENCH) 

3727 

49879 

506 . 12 

2 . 88 

0.25 

0 . 01 


Student-level Analysis with Achievement by Gender 

In our second example, we want to replicate another set of results presented in PIRLS 2011 
International Results in Reading. We are interested in investigating the relationship between fourth 
grade students’ gender and reading achievement. These results are presented in Exhibit 1.5 of PIRLS 
2011 International Results in Reading and repeated here in Exhibit 3.11. Because the results in this 
exhibit are based on plausible values, we must make sure they are included when creating the input 
file, and also indicate that this analysis will make use of achievement scores. 

After reviewing the appropriate codebook, the variable ITSEX in the student background data 
files contains information on the gender of students. We then proceed to read from the student 
background data files our variable of interest (ITSEX), the five plausible values of reading 
achievement (ASRREAO 1 through ASRREA05), the student sampling weight (TOTWGT), the 
variables that contain the jackknifing information (JKZONE and JKREP), and the country 
identification variable (IDCNTRY). Again, we will use the data of all available countries contained 
in the file ASGALLR3. 

The SAS program that implements this second example is presented in Exhibit 3.12 and is 
included in the International Database under the name EXAMPLE2.SAS. Note that one of the steps 
in this program is to select only those students who have non-missing data in our variable of interest 
ITSEX. 

The results obtained from this program are shown in Exhibit 3.13. For the sake of brevity, only 
the results of the first four countries are shown. The very same set of steps used in our first example 
apply here, adding ITSEX as a classification variable along with IDCNTRY. 
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Exhibit 3.1 1 : Exhibit of Example Student-level Analysis with Achievement by Gender, Taken from 
PIRLS 2011 International Results in Reading (Exhibit 1 .5) 


Exhibit 1 .5: Average Reading Achievement by Gender 


PIRLS 20111 



Girls 

Boys 

Difference 

Gender Difference 

Country 

Percent of 

Average Scale 

Percent of 

Average Scale 

(Absolute 

Girls 


Boys 


Students 

Score 

Students 

Score 

Value) 

Scored Higher 

Scored Higher 

Colombia 

49 (1.3) 

447 (4.6) 

51 (1.3) 

448 (4.6) 

1 (3.9) 







Italy 50(0.7) 

543 (2.4) 

50(0.7) 

540 (2.7) 

3 (2.4) 



i 




France 

49 (0.8) 

522 (3.4) 

51 (0.8) 

518 (2.4) 

5(2.7) 







Spain 

49 (0.8) 

516 (2.5) 

51 (0.8) 

511 (2.8) 

5(2.5) 







2 1 Belgium (French) 

49 (0.9) 

509 (3.1) 

51 (0.9) 

504 (3.1) 

5(2.3) 







3 Israel 

51 (1.6) 

544 (3.1) 

49(1.6) 

538 (3.4) 

6(3.4) 







Czech Republic 

49 (1.2) 

549 (2.5) 

51 (1.2) 

542 (2.5) 

6 (2.6) 







t Netherlands 

51 (0.7) 

549 (2.1) 

49 (0.7) 

543 (2.2) 

7 (2.0) 



■ 




Austria 

49 (1.2) 

533 (2.2) 

51 (1.2) 

525 (2.3) 

8 (2.3) 



■ 




Germany 

49 (0.8) 

545 (2.3) 

51 (0.8) 

537 (2.7) 

8 (2.5) 



■ 




Slovak Republic 

49 (0.8) 

540 (3.1) 

51 (0.8) 

530 (2.8) 

10 (2.1) 







2 United States 

51 (0.5) 

562 (1.9) 

49(0.5) 

551 (1.7) 

10(1.8) 







2 Denmark 

50 (0.7) 

560 (1.9) 

50 (0.7) 

548 (2.1) 

12(2.2) 







2 Canada 

49 (0.6) 

555 (1.7) 

51 (0.6) 

542 (2.1) 

12(2.0) 







Poland 

48 (0.9) 

533 (2.5) 

52 (0.9) 

519 (2.7) 

14(3.1) 







2 Azerbaijan 

47 (0.9) 

470 (3.6) 

53 (0.9) 

456 (3.5) 

14 (2.3) 







2 Croatia 

50 (0.8) 

560 (2.1) 

50 (0.8) 

546 (2.2) 

14(2.2) 







Sweden 

49 (1.0) 

549 (2.4) 

51 (1.0) 

535 (2.5) 

14(2.7) 







Portugal 

49 (1.2) 

548 (3.0) 

51 (1.2) 

534 (2.8) 

14 (2.4) 







* Norway 

52 (1.0) 

514 (2.2) 

48 (1.0) 

500 (2.7) 

14(3.1) 







Chinese Taipei 

47 (0.6) 

561 (2.1) 

53 (0.6) 

546 (2.1) 

15(2.1) 







Bulgaria 

49 (0.9) 

539 (4.5) 

51 (0.9) 

524 (4.3) 

15 (3.5) 







Romania 

48 (0.9) 

510 (4.8) 

52 (0.9) 

495 (4.3) 

15(3.3) 







Ireland 

49 (2.2) 

559 (2.9) 

51 (2.2) 

544 (3.0) 

15 (3.9) 



■■ 




Hungary 

49 (0.9) 

547 (3.2) 

51 (0.9) 

532 (3.2) 

16 (2.6) 







Slovenia 

48 (0.8) 

539 (2.2) 

52 (0.8) 

523 (2.7) 

16(3.1) 







t Northern Ireland 

50 (1.2) 

567 (2.5) 

50(1.2) 

550 (3.2) 

16(3.4) 







3 Hong Kong SAR 

46(1.2) 

579 (2.3) 

54(1.2) 

563 (2.5) 

16(2.2) 







Australia 

49 (1.1) 

536 (2.7) 

51 (1.1) 

519 (2.7) 

17(3.1) 







2 Singapore 

49 (0.6) 

576 (3.5) 

51 (0.6) 

559 (3.6) 

17(2.6) 







Malta 

49 (0.5) 

486(1.9) 

51 (0.5) 

468 (2.0) 

18 (2.8) 







Indonesia 

51 (0.9) 

437 (4.5) 

49 (0.9) 

419 (4.3) 

18(2.3) 







1 2 Lithuania 

48 (0.8) 

537 (2.4) 

52 (0.8) 

520 (2.4) 

18 (2.8) 







Russian Federation 

49 (1.0) 

578 (2.8) 

51 (1.0) 

559 (3.1) 

18(2.3) 







Iran, Islamic Rep. of 

49 (2.9) 

467 (4.3) 

51 (2.9) 

448 (4.3) 

20 (6.4) 







New Zealand 

49 (1.0) 

541 (2.2) 

51 (1.0) 

521 (2.7) 

20(3.1) 







Finland 

49 (0.8) 

578 (2.3) 

51 (0.8) 

558 (2.2) 

21 (2.3) 


i 





1 Georgia 

48 (0.9) 

499 (2.7) 

52(0.9) 

477 (4.0) 

22 (3.0) 


i 





t England 

49 (1.0) 

563 (3.0) 

51 (1.0) 

540 (3.1) 

23 (3.0) 






United Arab Emirates 

50 (1.6) 

452 (3.0) 

50 (1.6) 

425 (3.5) 

27 (4.8) 






>K Morocco 

48 (0.8) 

326 (4.0) 

52 (0.8) 

296 (4.6) 

29 (3.9) 






2 Qatar 

47 (3.4) 

441 (4.7) 

53 (3.4) 

411 (4.2) 

30 (6.0) 






Trinidad and Tobago 

49 (2.0) 

487 (4.5) 

51 (2.0) 

456 (4.3) 

31 (4.6) 






^ Oman 

49 (0.7) 

411 (3.0) 

51 (0.7) 

371 (3.4) 

40 (2.9) 






Saudi Arabia 

52(1.5) 

456 (3.1) 

48 (1.5) 

402 (8.2) 

54 (8.8) 






International Avg. 

49(0.2) 

520 (0.5) 

51 (0.2) 

504 (0.5) 

16(0.5) 






8 

1 1 

O 40 

C 

1 1 1 

» 40 81 


Girls 

Boys 

Difference 

Gender Difference 

Country 

Percent of 

Average Scale 

Percent of 

Average Scale 

(Absolute 

Girls 


Boys 


Students 

Score 

Students 

Score 

Value) 

Scored Higher 

Scored Higher 

Sixth Grade Participants 

Honduras 

51 (1.2) 

455 (5.5) 

49 (1.2) 

444 (5.0) 

12 (4.2) 


mm 



Botswana 

51 (0.8) 

432 (4.2) 

49(0.8) 

405 (4.8) 

28 (3.4) 





Morocco 

48 (0.7) 

443 (3.8) 

52 (0.7) 

408 (4.5) 

35 (3.5) 





i* Kuwait 54(1.9) 

443 (6.4) 

46(1.9) 391 (7.3) 

53 (9.3) 





Benchmarking Participants 0 

Andalusia, Spain 

50 (0.9) 

519 (2.4) 

50 (0.9) 

511 (2.8) 

8 (2.6) 


- 




2 Alberta, Canada 

48 (0.9) 553 (3.1) 

52 (0.9) 

543 (3.1) 10(2.2) 


■■ 




2 Ontario, Canada 

49(1.1) 

558 (3.3) 

51 (1.1) 

546 (2.8) 

13 (3.4) 






Dubai, UAE 47 (23) 

483 (3.9) 

53 (2.3) 470 (3.5) 13 (6.3) 






Quebec, Canada 

50 (1.0) 

544 (2.6) 

50 (1.0) 

531 (2.4) 

14 (2.5) 






1 3 Florida, US 

51 (0.9) 

576 (3.4) 

49(0.9) 

561 (3.0) 

15 (2.9) 






Maltese - Malta 

49 (0.5) 

470 (2.0) 

51 (0.5) 

445 (2.2) 

25 (3.0) 





v Eng/Afr (5) - RSA 

49(1.3) 

434 (7.7) 

51 (1.3) 

408 (8.7) 

26 (7.7) 





Abu Dhabi, UAE 

50 (2.9) 

442 (55) 

50 (2.9) 

406 (6.3) 

36 (8.0) 






0 Republic of South Africa (RSA) tested 5th grade students receiving instruction in English (ENG) or Afrikaans (AFR). ^ 

■ Difference statistically significant 

■ Difference not statistically significant 

>K Average achievement not reliably measured because the percentage of students with achievement too low for estimation exceeds 25%. 

V Reservations about reliability of average achievement because the percentage of students with achievement too low for estimation does not exceed 25% but exceeds 1 5%. 
See Appendix C.2 for target population coverage notes 1, 2, and 3. See Appendix C.5 for sampling guidelines and sampling participation notes t and 4. 

( ) Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 
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SOURCE: lEA's Progress in International Reading Literacy Study - PIRLS 201 1 





Exhibit 3.1 2: Example SAS Program for Student-level Analysis with Achievement by Gender (EXAMPLE2.SAS) 


LIBNAME Pll " <datpath> " ; 

% INCLUDE " <macpath> JACKPV . SAS 11 ; 

DATA ASGALLR3 ; 

SET Pll . ASGALLR3 ; 

WHERE NMISS (ITSEX) = 0 ; 

PROC FORMAT LIBRARY = WORK ; 

VALUE COUNTRY 

< list PIRLS 2011 country formats > ; 

VALUE SEX 

1 = "GIRL" 

2 = "BOY" ; 

% JACKPV (TOTWGT , JKZONE , JKREP, 75, IDCNTRY ITSEX, ASRREAO , 5, ASGALLR3 ) ; 

PROC PRINT DATA = FINAL NOOBS ; 

VAR IDCNTRY ITSEX N TOTWGT MNPV MNPV_SE PCT PCT_SE ; 

FORMAT IDCNTRY COUNTRY. ITSEX SEX. N 6.0 TOTWGT 10.0 
MNPV MNPV_SE PCT PCT_SE 6.2 ; 


Exhibit 3.1 3: Output for Example Student-level Analysis with Achievement by Gender (EXAMPLE 2) 


IDCNTRY 

ITSEX 

N 

TOTWGT 

MNPV 

MNPV_SE 

PCT 

PCT_SE 

AUSTRALIA 

GIRL 

3048 

123760 

535 . 79 

2.67 

49.26 

1 . 06 

AUSTRALIA 

BOY 

3078 

127453 

519.20 

2 . 73 

50 . 74 

1 . 06 

AUSTRIA 

GIRL 

2274 

38141 

532 . 76 

2 . 18 

48.69 

1 . 18 

AUSTRIA 

BOY 

2396 

40192 

525 . 19 

2.32 

51.31 

1 . 18 

AZERBAIJAN 

GIRL 

2241 

49037 

469.57 

3.56 

47 . 14 

0 . 85 

AZERBAIJAN 

BOY 

2640 

54977 

455 . 82 

3.47 

52 . 86 

0 . 85 

BELGIUM (FRENCH) 

GIRL 

1815 

24388 

508 . 85 

3 . 11 

48 . 89 

0 . 93 

BELGIUM (FRENCH) 

BOY 

1912 

25491 

503.51 

3 . 11 

51 . 11 

0 . 93 


In Exhibit 3.13, each country’s results are displayed on two lines, one for each value of the 
variable ITSEX. The countries are identified in the first column and the second colu mn describes the 
category of ITSEX being reported. The third colu mn reports the number of valid cases and the fourth 
the sum of weights of the sampled students. The next four columns report the estimated average 
reading achievement and its standard error, followed by the percentage of students in each category 
and its standard error. As shown in the first two lines, the average reading achievement was 535.79 
for fourth grade girls in Australia (standard error of 2.67) and 5 19.20 for fourth grade boys (standard 
error of 2.73). An estimated 49.26 percent (standard error of 1.06) of fourth grade students in 
Australia were girls, and 50.74 percent were boys (standard error of 1.06). 

3.8 PIRLS/prePIRLS Analyses with Home Background Data 

Analyzing home background data from the PIRLS 201 1 International Database requires that the 
home background data files (ASH) be merged with the student background data files (ASG) in order 
to retrieve the achievement scores and the required sample design variables. To illustrate how to 
perform analyses based on home background data, we will use an example that computes the average 
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scale score for one of the newly developed contextual scales, along with the percentages of students, 
with their average reading achievement, for the categories of the scale’s corresponding index. 

In general, to perform analyses with home background variables using the home background data 
files, you should do the following: 

1 . Identify the variables of interest in the home background data files and note any specific national 
adaptations to the variables. 

2. Retrieve the relevant variables from the home background data files, including analysis and 
classification variables, identification variables (IDCNTRY and IDSTUD), and any other 
variables used in the selection of cases. 

3. Retrieve the relevant variables from the student background data files, including plausible values 
for achievement, classification variables, identification variables (IDCNTRY and IDSTUD), 
sampling (JKZONE and JKREP) and weighting (TOTWGT) variables, and any other variables 
used in the selection of cases. 

4. Merge the home background data files with the student background data files using the variables 
IDCNTRY and IDSTUD. 

5. Perform any necessary variable transformations or recodes. 

6. Use the macros JACKGEN and JACKREG, or JACKPV and JACKREGP if plausible values are 
involved, with the appropriate parameters. 

7. Specify the location of the data files (<datpath>) and the macros (<macpath>). 

8. Print out the results file. 

Analyses with a Contextual Scale 

PIRLS 2011 has innovated in its reporting of contextual data by creating contextual scales based on 
Rasch modeling. A number of contextual scales were reported in PIRLS 2011 International Results 
in Reading and are available in the International Database for analysis. Each contextual scale 
variable is a Rasch scale with an international mean of 10 and an international standard deviation 
of 2. An index was derived from each scale that divides the range of scores into usually three 
categories: the most desirable scores (high values), the least desirable scores (low values), and the 
remaining scores in between. 

These contextual scales and their corresponding indices were reported in PIRLS 2011 
International Results in Reading. Exhibit 3.14 shows one such example, Exhibit 4.6 from PIRLS 
2011 International Results in Reading, analyzing students’ Early Literacy Activities Before 
Beginning Primary School as reported by their parents. Results on the Rasch scale are reported for 
each country as an “Average Scale Score” and its corresponding index is reported as the percentages 
of students in each category — Often, Sometimes, and Never or Almost Never — along with the 
average achievement in each category. The Rasch scores for this contextual scale are contained in the 
ASBHELA variable; the corresponding index is found in the ASDHELA variable. 


3 The contextual scales are described in the context questionnaire scales section of Methods and Procedures in TIMSS and PIRLS 2011. 
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Exhibit 3.14: Example Exhibit of a Contextual Scale, Taken from 

PIRLS 2011 International Results in Reading (Exhibit 4.6) 


Exhibit 4.6: Early Literacy Activities Before Beginning Primary School 

Reported by Parents 


PIRLS 20111 


Students were scored according to their parents' frequency of doing the nine activities on the Early Literacy Activities scale. Students Often engaged 
in early literacy activities had a score on the scale of at least 10.7, which corresponds to their parents "often" doing five of the nine activities with them 
and "sometimes" doing the other four, on average. Students Never or Almost Never engaged in such activities had a score no higher than 6.2, which 
corresponds to parents "never or almost never" doing five of the nine activities with them and "sometimes" doing the other four, on average. All other 
students had parents who Sometimes engaged them in early literacy activities. 


Often 


Sometimes 


Never or Almost Never 


Country 


Russian Federation 


Percent 
of Students 

61 (1.3) 


Average 

Achievement 

576 (2.7) 


Percent 
of Students 

38(1.2) 


Average 

Achievement 

558 (3.4) 


Percent 
of Students 

1 (0.3) 


Average 

Achievement 


Average 
Scale Score 


11.1 (0.06) 


Northern Ireland 

s 

59 (1.3) 

■■ 

582 (3.5) 

■■ 

41 (1.4) 

559(3.7) 

0(0.2) 

■■ 

~ ~ 

1 ■ 

11.2 (0.06) 

New Zealand 

s 

55 (1.0) 


567 (2.7) 


44(1.0) 

529 (2.5) 

1 (0.1) 


~~ 


11.0(0.05) 

Australia 

S 

52 (1.4) 

■■ 

555 (3.0) 

■■ 

46(1.3) 

528 (3.4) 

1 (0.3) 

■■ 

— 

1 I 

10.8 (0.06) 

Georgia 


52 (1.4) 


498 (2.6) 


47(1.3) 

479 (4.0) 

1 (0.2) 


~~ 


10.7 (0.06) 

Canada 

r 

51 (0.9) 

■■ 

566 (1.9) 

■■ 

48 (0.9) 

541 (1.8) 

1 (0.1) 

■■ 

— 

1 ■ 

10.7 (0.04) 

Ireland 


50 (0.9) 


569 (2.3) 


49 (0.8) 

542 (2.6) 

1 (0.1) 


~~ 


10.8 (0.04) 

Croatia 

■■ 

50 (0.9) 

■■ 

562 (2.2) 

■■ 

49 (0.9) 

544(1.9) 

0(0.1) 

■■ 

— 

1 ■ 

10.7 (0.03) 

Slovenia 


48 (1.2) 


543 (2.3) 


51 (1.2) 

522 (2.6) 

0(0.1) 




10.6 (0.04) 

Israel 

■■ 

48 (1.0) 

■■ 

563 (3.0) 

■■ 

51 (1.0) 

534(3.5) 

1 (0.2) 

■■ 


1 1 

10.6 (0.04) 

Italy 


48 (0.9) 


553 (2.4) 


51 (1.0) 

537 (2.6) 

1 (0.2) 




10.5 (0.03) 

Slovak Republic 

■■ 

47 (0.9) 

■■ 

547 (2.9) 

■■ 

51 (0.9) 

530 (2.5) 

2(0.6) 

■■ 

— 

1 I 

10.5 (0.05) 

Trinidad and Tobago 


47(1.1) 


497 (4.0) 


52 (1.1) 

456 (4.1) 

1 (0.3) 


~~ 


10.5 (0.05) 

Malta 

■■ 

45 (0.9) 

■■ 

507 (1.9) 

■■ 

54 (0.9) 

463 (2.7) 

1 (0.2) 

■■ 

— 

1 I 

10.4 (0.04) 

Spain 


44 (1.0) 


528 (2.7) 


55 (1.0) 

507 (2.7) 

1 (0.2) 


~~ 


10.4 (0.03) 

Poland 

■■ 

43 (0.8) 

■■ 

544 (2.8) 

■■ 

56 (0.8) 

514(2.1) 

1 (0.3) 

■■ 


1 1 

10.4 (0.03) 

Hungary 


43 (0.8) 


553 (2.8) 


56 (0.8) 

535 (3.2) 

1 (0.4) 


~~ 


10.3 (0.04) 

Czech Republic 

■■ 

40 (1.0) 

■■ 

555 (2.6) 

■■ 

60(1.0) 

542(2.3) 

1 (0.2) 

■■ 


1 1 

10.3 (0.03) 

Netherlands 

s 

40 (0.8) 


559 (3.1) 


60 (0.8) 

551 (2.0) 

1 (0.2) 




10.2 (0.03) 

Bulgaria 

■■ 

39 (1.4) 

■■ 

559 (3.1) 

■■ 

51 (1.0) 

529 (3.7) 

9(1.4) 

■■ 

455 (15.3) 

1 ■ 

9.7 (0.12) 

Romania 


38 (1.5) 


529 (4.1) 


54(1.3) 

494 (4.5) 

8(1.0) 


423 (8.9) 


9.9 (0.09) 

Germany 

r 

38 (0.9) 

■■ 

555 (2.8) 

■■ 

61 (0.9) 

543 (2.2) 

1 (0.2) 

■■ 

— 

1 ■ 

10.2 (0.03) 

Norway 


37(1.4) 


524 (2.5) 


63 (1.4) 

500 (2.2) 

1 (0.2) 


~~ 


10.0 (0.06) 

France 

■■ 

36 (0.7) 

■■ 

536 (2.6) 

■■ 

63 (0.7) 

515(2.7) 

1 (0.2) 

■■ 

— 

1 I 

10.0 (0.03) 

Lithuania 


36 (0.9) 


541 (1.9) 


63 (0.9) 

524 (2.5) 

2(0.2) 


~~ 


10.0 (0.03) 

Austria 

■■ 

35 (1.0) 

■■ 

543 (2.1) 

■■ 

63 (1.1) 

523 (2.5) 

1 (0.2) 

■■ 

— 

1 ■ 

10.0 (0.03) 

Portugal 


35(1.1) 


558 (2.8) 


63(1.1) 

535 (2.6) 

2(0.4) 


~~ 


10.0 (0.05) 

Sweden 

■■ 

34 (1.0) 

■■ 

562 (2.9) 

■■ 

64(1.0) 

537 (2.2) 

2 (0.2) 

■■ 

— 

1 ■ 

9.9 (0.04) 

Colombia 


34 (1.1) 


457 (5.7) 


63 (1.0) 

448 (3.8) 

3 (0.4) 


409(11.0) 


9.9 (0.06) 

Denmark 

■■ 

32 (0.9) 

■■ 

567 (2.2) 

■■ 

67 (0.9) 

550(1.9) 

1 (0.2) 

■■ 

— 

1 I 

9.9 (0.03) 

Belgium (French) 


30 (0.8) 


524 (2.9) 


67 (0.7) 

501 (3.2) 

3 (0.4) 


482 (10.3) 


9.7 (0.04) 

Qatar 

■■ 

28 (1.0) 

■■ 

458 (6.0) 

■■ 

69 (1.0) 

420 (3.1) 

3 (0.3) 

■■ 

390 (11.7) 

1 ■ 

9.7 (0.05) 

Finland 


27 (0.9) 


583 (2.9) 


72 (0.9) 

564 (1.9) 

1 (0.1) 




9.7 (0.03) 

United Arab Emirates 

■■ 

27 (0.5) 

■■ 

480 (2.8) 

■■ 

71 (0.5) 

430 (2.3) 

3 (0.2) 

■■ 

392 (7.4) 

1 ■ 

9.6 (0.03) 

Saudi Arabia 


26 (1.3) 


455 (5.2) 


70(1.4) 

426 (4.2) 

4(0.8) 


360 (16.2) 


9.5 (0.06) 



26 (0.7) 


595 (3.6) 

■■ 

69 (0.7) 

561 (3.3) 

5(0.3) 

■■ 

543 (6.2) 

1 1 

9.4 (0.03) 

Azerbaijan 


23 (1.3) 


467 (4.8) 


72(1.3) 

463 (3.5) 

4(0.7) 


439 (7.6) 


9.5 (0.07) 

Indonesia 

■■ 

23 (1.3) 

■■ 

445 (5.8) 

■■ 

72(1.3) 

427 (4.1) 

4(0.5) 

■■ 

409 (6.5) 

1 1 

9.4 (0.07) 

Oman 


19 (0.4) 


429 (3.6) 


76 (0.5) 

385 (3.2) 

5 (0.4) 


354 (7.2) 


9.2 (0.03) 

Morocco 

■■ 

17 (0.9) 

■■ 

321 (5.6) 

■■ 

64(1.5) 

314 (4.5) 

19 (1.9) 

■■ 

302 (13.6) 

1 ■ 

8.4 (0.13) 

Iran, Islamic Rep. of 


15 (0.6) 


474 (3.6) 


77 (0.8) 

460 (3.0) 

8 (0.8) 


411 (8.0) 


8.9 (0.05) 

Chinese Taipei 


14 (0.6) 

■■ 

577 (3.5) 

■■ 

76 (0.8) 

553 (1.8) 

10 (0.6) 

■■ 

526 (4.8) 

1 ■ 

8.7 (0.04) 

Hong Kong SAR 


12 (0.6) 


588 (3.7) 


80 (0.6) 

571 (2.3) 

8(0.5) 


560 (3.7) 


8.7 (0.03) 

International Avg. 


37(0.2) 


529 (05) 


60 (0.2) 

506(0.5) 

3(0.1) 


430 (2.6) 

1 


Sixth Grade Participants 













Honduras 


30 (1.5) 


461 (8.3) 


63 (1.3) 

447 (4.2) 

7 (0.7) 


426 (12.2) 


9.6 (0.08) 

Kuwait 

5 

23 (1.2) 

■■ 

448 (9.3) 

■■ 

73 (1.2) 

420 (6.5) 

4(0.5) 

■■ 

378 (14.2) 

1 1 

9.4 (0.06) 

Morocco 


16 (0.7) 


437 (6.7) 


67 (1.2) 

432 (4.2) 

16(1.3) 


406 (7.5) 


8.5 (0.08) 

Botswana 

■■ 

15(1.1) 

■■ 

464 (8.7) 

■■ 

74(1.2) 

420 (3.9) 

11 (1.0) 

■■ 

388 (6.2) 

■ ■ 

8.6 (0.08) 

Benchmarking Participants 0 

Ontario, Canada 

r 

54(1.3) 


566 (3.1) 


45 (1.3) 

545 (3.2) 

1 (0.2) 




10.9 (0.05) 

Alberta, Canada 

r 

52(1.3) 

■■ 

568 (3.6) 

■■ 

47 (1.3) 

542 (3.7) 

1 (0.2) 

■■ 

— 

1 ■ 

10.8 (0.05) 

Maltese - Malta 


45 (0.8) 


481 (2.0) 


54 (0.8) 

447 (1.9) 

1 (0.2) 




10.4 (0.03) 

Andalusia, Spain 

■■ 

43 (0.9) 

■■ 

530 (2.5) 

■■ 

56 (0.9) 

508 (2.7) 

1 (0.2) 

■■ 

— 

1 ■ 

10.3 (0.04) 

Quebec, Canada 


40(1.0) 


554 (3.0) 


59 (1.0) 

531 (2.4) 

2 (0.3) 




10.2 (0.04) 

Eng/Afr (5) - RSA 

r 

34(1.5) 

■■ 

465 (8.5) 

■■ 

61 (1.4) 

405 (7.2) 

4(0.7) 

■■ 

358 (14.5) 

1 ■ 

9.8 (0.09) 

Dubai, UAE 


34 (0.8) 


521 (2.4) 


64 (0.8) 

463 (2.6) 

2(0.2) 




9.9 (0.03) 

Abu Dhabi, UAE 

■■ 

24(1.1) 

■■ 

466 (6.4) 

■■ 

73 (1.0) 

416 (4.4) 

3 (0.4) 

■■ 

379 (9.6) 

1 ■ 

9.5 (0.05) 


0 Republic of South Africa (RSA) tested 5th grade students receiving instruction in English (ENG) or Afrikaans (AFR). 

England and the United States did not administer the Home Questionnaire. 

Centerpolnt of scale set at 1 0. 

( ) Standard errors appear in parentheses. Because of rounding some results may appear inconsistent 
A tilde (~) indicates insufficient data to report achievement. 

An "i" indicates data are available for at least 70% but less than 85% of the students. An "s" Indicates data are available for at least 50% but less than 70% of 
the students. 
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This example will replicate both the average scale score of the Early Literacy Activities 
contextual scale and the percentages of students, with their average reading achievement, in each 
category of its index. This will be done in two steps, both using the merged ASHALLR3 data file 
that combines the home background data files and the student background data files. 


The first step, our third example, will compute the average scale score using the contextual scale 
variable ASBHELA. After reviewing the appropriate codebook, the variable ASBHELA in the home 
background data files contains information on the Early Literacy Activities Before Beginning 
Primary School contextual scale. We then proceed to read from the student background data files our 
variable of interest (ASBHELA), the student sampling weight (TOTWGT), the variables that contain 
the jackknifing information (JKZONE and JKREP), and the country identification variable 
(IDCNTRY). We will use the data of all available countries contained in the merged file 
ASHALLR3. 

The SAS program that implements our third example is presented in Exhibit 3.15 and is included 
in the International Database under the name EXAMPLE3.SAS. Note that one of the steps in this 
program is to select only those students who have non-missing data in our variable of interest 
ASBHELA. 


Exhibit 3.1 5: Example SAS Program for Student-level Analysis with a Contextual Scale (EXAMPLE3.SAS) 


LIBNAME Pll " <datpath> " ; 

% INCLUDE " <macpath> JACKGEN . SAS " ; 

PROC SORT DATA = Pll . ASHALLR3 OUT = ASHALLR3 ; 

BY IDCNTRY IDSTUD ; 

PROC SORT DATA = Pll . ASGALLR3 OUT = ASGALLR3 ; 

BY IDCNTRY IDSTUD ; 

DATA ASHMERGED ; 

MERGE ASHALLR3 (IN = INASH) 

ASGALLR3 (IN = INASG) ; 

BY IDCNTRY IDSTUD ; 

IF INASH AND INASG ; 

DATA ASHMERGED ; 

SET ASHMERGED ; 

WHERE NMISS (ASBHELA) = 0 ; 

PROC FORMAT LIBRARY = WORK ; 

VALUE COUNTRY 

< list PIRLS 2011 country formats > ; 

% JACKGEN (TOTWGT, JKZONE, JKREP, 75, IDCNTRY, ASBHELA, ASHMERGED) ; 

PROC PRINT DATA = FINAL NOOBS ; 

VAR IDCNTRY N TOTWGT MNX MNX_SE PCT PCT_SE ; 

FORMAT IDCNTRY COUNTRY. N 6.0 TOTWGT 10.0 
MNX MNX_SE PCT PCT_SE 6.2 ; 


The results obtained from this program are shown in Exhibit 3.16. For the sake of brevity, only 
the results of the first four countries are shown. In the first line of the results in Exhibit 3. 16, we see 
that students in Australia scored 10.84, with a standard error of 0.06, on the Early Literacy Activities 
contextual scale. Note that this is well above the international average of 10, the difference being 
statistically significant at the 95% confidence level. 
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Exhibit 3.1 6: Output for Example Student-level Analysis with a Contextual Scale (EXAMPLE 3) 


IDCNTRY 

N 

TOTWGT 

MNX 

MNX_SE 

PCT 

PCT_SE 

AUSTRALIA 

3232 

135420 

10 . 84 

0 . 06 

0 . 92 

0 . 03 

AUSTRIA 

4393 

74088 

9 . 98 

0 . 03 

0.50 

0 . 02 

AZERBAIJAN 

4509 

96026 

9.47 

0 . 07 

0.65 

0 . 03 

BELGIUM (FRENCH) 

3383 

44793 

9.69 

0 . 04 

0.30 

0 . 01 


In the second step, our fourth example, we will compute the percentage of fourth grade students, 
with their average reading achievement, in each category of the index variable ASDHELA. This 
analysis is similar to our earlier example of a student-level analysis with achievement by gender. Our 
variable of interest here is the ASDHELA index variable of our contextual scale and it will be used 
to measure average reading achievement using the five plausible values ASRREA01 through 
ASRREA05. 

The SAS program that implements this fourth example is presented in Exhibit 3.17 and is 
included in the International Database under the name EXAMPLE4.SAS. Note that one of the steps 
in this program is to select only those students who have non-missing data in our variable of interest 
ASDHELA. The results obtained from this program are shown in Exhibit 3.18. For the sake of 
brevity, only the results of the first four countries are shown. 

Exhibit 3.1 7: Example SAS Program for Student-level Analysis with a Contextual Scale (EXAMPLE4.SAS) 


LIBNAME Pll " <datpath> " ; 

% INCLUDE " <macpath> JACKPV . SAS " ; 

PROC SORT DATA = Pll . ASHALLR3 OUT = ASHALLR3 ; 

BY IDCNTRY IDSTUD ; 

PROC SORT DATA = Pll . ASGALLR3 OUT = ASGALLR3 ; 

BY IDCNTRY IDSTUD ; 

DATA ASHMERGED ; 

MERGE ASHALLR3 (IN = INASH) 

ASGALLR3 (IN = INASG) ; 

BY IDCNTRY IDSTUD ; 

IF INASH AND INASG ; 

DATA ASHMERGED ; 

SET ASHMERGED ; 

WHERE NMISS (ASDHELA) = 0 ; 

PROC FORMAT LIBRARY = WORK ; 

VALUE COUNTRY 

< list PIRLS 2011 country formats > ; 

VALUE ELA 

1 = "OFTEN" 

2 = "SOMETIMES" 

3 = "NEVER OR ALMOST NEVER" ; 

% JACKPV (TOTWGT , JKZONE , JKREP , 75, IDCNTRY ASDHELA, ASRREAO , 5, ASHMERGED) ; 

PROC PRINT DATA = FINAL NOOBS ; 

VAR IDCNTRY ASDHELA N TOTWGT MNPV MNPV_SE PCT PCT_SE ; 

FORMAT IDCNTRY COUNTRY. ASDHELA ELA. N 6.0 TOTWGT 10.0 
MNPV MNPV_SE PCT PCT_SE 6.2 ; 
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As shown in the first three lines of the results, 52.34 percent of students in Australia often 
experienced early literacy activities before beginning primary school (standard error of 1.39), and 
their average reading achievement was 554.83 (standard error of 3.01); 46.23 percent sometimes 
experienced early literacy activities (standard error of 1.31), and their average reading achievement 
was 527.72 (standard error of 3.37); and 1.44 percent of students never or almost never experienced 
early literacy activities (standard error of 0.32), and their average reading achievement was 490.46 
(standard error of 23.75). The large standard error of this last estimated average achievement, based 
on a very small sample and accounting for less than 2 percent of the target population, should be a 
note of caution as to the poor reliability of that estimate. 

Exhibit 3.1 8: Output for Example Student-level Analysis with a Contextual Scale (EXAMPLE 4) 


IDCNTRY 

ASDHELA 


N 

TOTWGT 

MNPV 

MNPV_SE 

PCT 

PCT_SE 

AUSTRALIA 

OFTEN 


1745 

70874 

554 . 83 

3 . 01 

52.34 

1.39 

AUSTRALIA 

SOMETIMES 


1450 

62600 

527 . 72 

3.37 

46.23 

1.31 

AUSTRALIA 

NEVER OR ALMOST 

NEVER 

37 

1947 

490.46 

23 . 75 

1.44 

0.32 

AUSTRIA 

OFTEN 


1592 

26099 

543.39 

2 . 10 

35.23 

1 . 05 

AUSTRIA 

SOMETIMES 


2732 

46913 

523.49 

2.50 

63.32 

1 . 07 

AUSTRIA 

NEVER OR ALMOST 

NEVER 

69 

1077 

501 . 83 

7 . 79 

1.45 

0.21 

AZERBAIJAN 

OFTEN 


1119 

22528 

466 . 95 

4 . 84 

23.46 

1.33 

AZERBAIJAN 

SOMETIMES 


3206 

69288 

463.25 

3 .47 

72 . 16 

1.35 

AZERBAIJAN 

NEVER OR ALMOST 

NEVER 

184 

4211 

439.24 

7.56 

4.38 

0.69 

BELGIUM (FRENCH) 

OFTEN 


1029 

13472 

524.35 

2 . 95 

30 . 08 

0 . 76 

BELGIUM (FRENCH) 

SOMETIMES 


2263 

29978 

501 . 06 

3 . 16 

66 . 93 

0 . 72 

BELGIUM (FRENCH) 

NEVER OR ALMOST 

NEVER 

91 

1343 

482 . 03 

10.29 

3 . 00 

0.39 


3.9 PIRLS/prePIRLS Analyses with Teacher-level Variables 

The teachers in the PIRLS 2011 International Database do not constitute representative samples of 
teachers in the participating countries. Rather, they are the teachers of nationally representative 
samples of students. Therefore, analyses with teacher data should be made with students as the units 
of analysis and reported in terms of students who are taught by teachers with a particular attribute. 

When analyzing teacher data, it is first necessary to link the students to their respective teachers. 
The student-teacher linkage data files (AST) were created for this purpose. Student achievement 
scores (plausible values), jackknife replication information, and teacher weighting variables 
appropriate for conducting analyses with teacher variables are found in the student-teacher linkage 
data files in order to simplify the merging process for analyses that link teacher background variables 
to student achievement. For such analyses, it only is necessary to merge the teacher background data 
files (ATG) with the student-teacher linkage data files. For analyses linking teacher variables to 
student background variables, it also is necessary to merge the student background data files (ASG) 
with the teacher background data files, after combining the latter with the student-teacher linkage 
data files. 
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In general, to perform analyses using the teacher background data files, users should do the 
following: 

1 . Identify the variables of interest in the teacher background data files and note any specific 
national adaptations to the variables. 

2. Retrieve the relevant variables from the teacher background data files, including analysis 
variables, classification variables, identification variables (IDCNTRY, IDTEACH, and IDLINK), 
and any other variables used in the selection of cases. 

3. Retrieve the relevant variables from the student-teacher linkage data files, including plausible 
values of achievement, identification variables (IDCNTRY, IDSTUD, IDTEACH, and IDLINK), 
sampling (JKZONE and JKREP) and weighting (TCHWGT) variables, and any other variables 
used in the selection of cases. 

4. Merge the teacher background data files with the student-teacher linkage data files using the 
variables IDCNTRY, IDTEACH, and IDLINK. 

5. If student background variables also are needed, merge the student background data files with the 
merged student-teacher data files from the previous step using the variables IDCNTRY and 
IDSTUD. 

6. Perform any necessary variable transformations or recodes. 

7. Use the macros JACKGEN and JACKREG, or JACKPV and JACKREGP if plausible values are 
involved, with the appropriate parameters. 

8. Specify the location of the data files (<datpath>) and the macros (<macpath>). 

9. Print the results file. 

As our example of an analysis using teacher background data, we will investigate the years of 
experience of the PIRLS 2011 teachers. The results of such an analysis are presented in Exhibit 7.3 
of PIRLS 2011 International Results in Reading and are reproduced here in Exhibit 3.19. 

Conducting analyses with teacher data requires a few extra steps. As before, we first proceed to 
identify the variables relevant to the analysis in the appropriate files, and review the documentation 
for any specific national adaptations to the questions of interest (see Supplements 1 and 2 to this 
User Guide). Because we are using a teacher-level variable, we must use the teacher background data 
files and the student-teacher linkage data files to find the variables. From the teacher background 
data files, we need the variable that contains the information on the teachers’ years of experience 
(ATDG01), the variable that identifies the country (IDCNTRY), and the two teacher identification 
variables (IDTEACH and IDLINK) that will allow us to link the teacher data to the student data. 4 

We then proceed to retrieve the necessary information from the student-teacher linkage data 
files. From these files, we need the country identification (IDCNTRY) and the two teacher 
identification variables (IDTEACH and IDLINK) to link the teacher data to the student data. We also 
need the jackknife replication variables (JKZONE and JKREP), the teacher weighting variable 
(TCHWGT), and the reading achievement plausible values (ASRREA01 through ASRREA05). 


4 The information about teachers’ years of experience found in the ATBG01 variable was collapsed into reporting categories and stored in the 
derived variable ATDG01 (see Supplement 3 to this User Guide). 
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Exhibit 3.19: Exhibit of Example Teacher-level Analysis, Taken from 
PIRLS 2011 International Results in Reading (Exhibit 7.3) 


Exhibit 7.3: Teachers' Years of Experience 


PIRLS 2011 


4 th 

Grade 


Reported by Teachers 


Country 


20 Years or More 


At Least 10 but Less 
than 20 Years 


At Least 5 but Less 
than 10 Years 

Less than 5 Years 

Average 
Years of 
Experience 


Percent of 
Students 

Average 

Achievement 


Percent of 
Students 


Average 

Achievement 


Percent of 
Students 


Average 

Achievement 

Percent of 
Students 


Average 

Achievement 

Australia 

r 


42 (4.0) 

530 (5.1) 


22(3.5) 


533 (5.7) 


19 (2.9) 


529 (6.3) 

17(3.2) 


534 (7.2) 

17 (1.0) 

Austria 

■ 

I 

55 (2.8) 

532(2.4) 

II 

25(2.7) 

II 

526 (4.0) 

II 

11 (1.9) 

II 

532 (3.9) 

10 (1.9) 

II 

513 (6.8) 

21 (0.6) 

Azerbaijan 



60 (4.5) 

466 (3.8) 


26(3.1) 


456 (8.6) 


11 (2.7) 


440 (10.4) 

4(2.0) 


484 (33.4) 

23 (1.1) 

Belgium (French) 

■ 

1 

40 (3.9) 

516(3.9) 

II 

32(3.8) 

502 (4.9) 

II 

16 (3.4) 

II 

504(6.7) 

12 (2.5) 

II 

498 (12.5) 

16 (0.7) 

Bulgaria 



72 (3.3) 

533 (5.2) 


24 (3.0) 


532 (7.8) 


2(1.0) 



2 (12) 



24 (0.6) 

Canada 

■ 

1 

30(2.2) 

546(2.5) 

II 

33(2.6) 

II 

552 (3.7) 

II 

25 (1.9) 

II 

547 (3.4) 

12 (1.3) 

II 

545 (4.3) 

14 (0.4) 

Chinese Taipei 



26(3.3) 

558 (3.9) 


50 (3.8) 


551 (2.5) 


17(3.3) 


559 (5.1) 

7(2.0) 


536 (6.1) 

15 (0.6) 

Colombia 

■ 

I 

43 (4.5) 

447 (6.0) 

II 

35 (4.4) 

II 

442 (5.9) 

II 

16(3.3) 

II 

451 (15.8) 

7(1.6) 

II 

496 (19.4) 

18 (0.7) 

Croatia 



56 (3.4) 

557(2.5) 


30 (2.9) 


545 (3.2) 


9(2.0) 


559 (6.1) 

5 (1.4) 


552 (6.5) 

21 (0.7) 

Czech Republic 

■ 

1 

48 (4.0) 

543 (3.3) 

II 

27(3.6) 

II 

544 (3.5) 

II 

12 (2.4) 

II 

551 (5.4) 

13 (2.9) 

II 

554(7.7) 

18 (0.8) 

Denmark 



35 (3.7) 

557 (2.4) 


25(3.1) 


552 (3.6) 


22 (2.6) 


554 (2.5) 

18 (2.8) 


553 (4.5) 

16 (0.9) 

tngland 



14 (3.1) 

566 (8.0) 

II 

27(3.7) 

II 

550 (6.3) 

II 

29(3.7) 

II 

558(5.0) 

30 (4.1) 

II 

538(5.7) 

10 (0.7) 

Finland 



40 (3.1) 

567 (3.1) 


35 (3.1) 


570 (2.5) 


12 (2.0) 


571 (4.4) 

13 (2.0) 


564 (4.2) 

17 (0.6) 

France 

■ 

I 

34 (3.4) 

530 (3.7) 

II 

36 (3.0) 

II 

516 (3.8) 

II 

19(2.5) 

II 

520 (3.5) 

11 (2.1) 

II 

506 (8.0) 

16 (0.7) 

Georgia 



59 (3.4) 

486 (3.3) 


29(3.5) 


489 (6.2) 


8(1.9) 


497 (18.8) 

4(1.6) 


471 (16.4) 

22 (0.7) 

Germany 

■ 

i 

44 (3.8) 

539 (3.6) 

II 

25 (3.2) 

II 

542 (4.8) 

II 

14(2.7) 

II 

546 (6.4) 

16 (2.6) 

II 

540 (5.4) 

19 (0.9) 

Hong Kong SAR 



21 (3.5) 

564 (6.4) 


53 (4.0) 


569 (3.8) 


15 (3.3) 


573 (5.9) 

12 (2.5) 


582 (5.1) 

14 (0.7) 

Hungary 

■ 

l 

73 (3.0) 

544(3.1) 

II 

16 (2.8) 

II 

525 (12.8) 

II 

7 (1.7) 

II 

537 (11.7) 

4(1.4) 

II 

505 (10.9) 

24 (0.6) 

Indonesia 

r 


52 (4.9) 

438 (5.4) 


16(3.7) 


432 (10.3) 


19 (3.8) 


429 (10.6) 

12 (3.1) 


395 (15.3) 

18 (1.0) 

Iran, Islamic Rep. of 


1 

41 (3.6) 

476(5.3) 

II 

41 (3.5) 

II 

449 (5.0) 

II 

10 (1.9) 

II 

447 (13.3) 

9(1.8) 

II 

421 (10.8) 

17 (0.6) 

Ireland 



24 (3.0) 

551 (4.5) 


21 (3.1) 


555 (5.7) 


27(3.3) 


550 (4.0) 

27 (2.9) 


553 (4.6) 

12 (0.7) 

Israel 

■ 

1 

30 (3.8) 

545 (5.7) 

II 

36 (3.9) 

II 

546 (7.0) 

II 

16(2.7) 

II 

537 (9.6) 

18 (3.0) 

II 

531 (11.3) 

15 (0.8) 

Italy 



69 (3.6) 

543 (2.8) 


24 (3.4) 


539 (3.4) 


6(1.7) 


539 (7.4) 

1 (0.8) 


~~ 

24 (0.7) 

Lithuania 

■ 

1 

71 (2.5) 

527 (2.6) 

II 

26(2.3) 

II 

534 (3.6) 

II 

2(1.0) 

II 

— 

1 (0.5) 

II 

— 

24 (0.6) 

Malta 



18 (0.1) 

491 (2.8) 


38 (0.1) 


475 (2.5) 


29 (0.1) 


472 (2.4) 

15 (0.1) 


477 (4.1) 

12 (0.0) 

Morocco 

■ 

1 

56 (4.2) 

309 (5.3) 

II 

31 (4.4) 

II 

294 (8.9) 

II 

8(1.6) 

II 

336 (17.4) 

5 (1.4) 

II 

394 (16.4) 

21 (0.6) 

Netherlands 



28 (3.1) 

549 (3.0) 


29 (3.4) 


542 (3.2) 


24(3.2) 


549 (3.4) 

19 (3.0) 


545 (5.3) 

14 (0.8) 

New Zealand 

■ 

l 

20(2.5) 

542 (5.3) 

II 

26 (2.6) 

II 

533 (5.1) 

II 

28(2.5) 

II 

540 (4.6) 

27(2.5) 

II 

521 (5.4) 

11 (0.6) 

Northern Ireland 

r 


34 (4.7) 

556 (3.8) 


36 (4.0) 


563 (4.8) 


24 (4.2) 


561 (6.2) 

7(2.3) 


564 (20.2) 

16 (1.0) 

Norway 


1 

31 (4.4) 

506 (4.1) 

II 

40 (4.6) 

II 

510 (2.7) 

II 

15 (3.3) 

II 

507 (4.9) 

15 (2.4) 

II 

509 (5.9) 

16 (1.0) 

Oman 



12(1.9) 

409 (6.7) 


36 (2.8) 


3% (4.6) 


31 (2.9) 


385 (5.2) 

21 (2.1) 


385 (6.4) 

11 (0.4) 

Poland 

■ 

1 

83 (2.2) 

526(2.4) 

II 

11 (2.1) 

II 

529 (7.4) 

II 

4(1.5) 

II 

518 (10.7) 

2 (0.9) 

II 

-- 

23 (0.4) 

Portugal 



36 (3.2) 

549 (3.4) 


46 (3.6) 


532 (4.2) 


14(2.5) 


535 (6.5) 

4(1.6) 


566 (11.9) 

17 (0.6) 

Qatar 

■ 

1 

20 (4.3) 

450 (14.6) 

II 

25(4.3) 

II 

447 (9.1) 

II 

30 (4.1) 

II 

422 (8.6) 

25 (3.5) 

II 

388 (9.0) 

11 (0.7) 

Romania 



57(3.7) 

511 (4.8) 


31 (3.5) 


487 (8.2) 


9(2.3) 


478 (13.4) 

2 (1.0) 


~~ 

23 (0.8) 

Russian Federation 

■ 

1 

73 (3.0) 

571 (3.0) 

II 

22 (2.7) 

II 

567 (6.3) 

II 

3(1.1) 

II 

526 (16.2) 

3 (1.5) 

II 

559 (12.0) 

24 (0.7) 

Saudi Arabia 



16 (3.1) 

422 (14.5) 


50 (4.5) 


439 (5.3) 


18 (3.0) 


428 (15.2) 

17 (3.5) 


412 (12.9) 

13 (0.7) 

HSB999I 

■ 

1 

17 (1.8) 

570 (7.4) 

II 

27 (2.6) 

II 

563 (6.9) 

II 

24 (2.5) 

II 

575 (5.5) 

32 (2.1) 

II 

564 (5.7) 

11 (0.5) 

Slovak Republic 



55 (3.0) 

536 (4.4) 


28 (2.9) 


531 (3.4) 


10(2.1) 


546 (5.9) 

7(1.8) 


530 (8.4) 

20 (0.5) 

Slovenia 

■ 

1 

57 (3.8) 

532 (2.3) 

II 

27 (3.1) 

II 

532 (3.9) 

II 

10(2.2) 

II 

517 (6.4) 

6(1.5) 

II 

523 (8.0) 

21 (0.7) 

Spain 



59 (4.1) 

519 (3.0) 


19 (3.4) 


502 (7.1) 


8(1.3) 


510 (7.3) 

14 (2.6) 


502 (6.9) 

21 (0.8) 

Sweden 

■ 

1 

29 (4.2) 

546 (3.8) 

II 

45 (4.0) 

II 

543 (3.8) 

II 

18(2.7) 

II 

529 (4.3) 

8(1.9) 

II 

551 (6.3) 

16 (0.9) 

Trinidad and Tobago 



43 (4.0) 

487 (6.5) 


35 (4.0) 


459 (7.1) 


9(2.4) 


459 (15.4) 

12 (2.5) 


452 (13.8) 

18 (1.0) 

United Arab Emirates 

■ 

l 

12(1.6) 

434(9.1) 

II 

30 (2.3) 

II 

441 (7.3) 

II 

33 (2.3) 

II 

439 (6.2) 

25 (2.0) 

II 

436 (6.0) 

10 (0.3) 

United States 



28 (2.2) 

569 (3.8) 


38 (2.1) 


553 (3.1) 


19 (2.0) 


550 (4.3) 

15 (1.9) 


552 (5.2) 

14 (0.5) 

International Avg. 



41 (0.5) 

517 (0.8) 


31 (0.5) 


511 (0.9) 


16 (0.4) 


510(1.4) 

12 (03) 


507(1.7) 

17 (0.1) 

Sixth Grade Participants 

Botswana 



29 (4.0) 

434 (9.2) 


32 (4.1) 


421 (10.4) 


23 (3.7) 


401 (8.4) 

16 (3.0) 


420 (9.8) 

14 (0.8) 

Honduras 

■ 

1 

29(4.2) 

465 (6.4) 

II 

37(4.6) 

II 

436 (7.9) 

II 

17 (3.7) 

II 

458 (7.2) 

17 (4.0) 

II 

459 (20.6) 

14(0.9) 

Kuwait 

s 


9(2.9) 

419 (7.2) 


23 (4.5) 


412 (16.5) 


16 (3.8) 


429 (21.7) 

52 (4.9) 


419 (12.8) 

7(0.7) 

Morocco 

r 

l 

53 (4.4) 

422 (7.4) 

II 

38(4.3) 

II 

407 (9.3) 

II 

7 (2.0) 

II 

468 (14.8) 

2(0.9) 

II 

~~ 

21 (0.7) 

Benchmarking Participants 0 

Alberta, Canada 



33 (3.8) 

544 (4.2) 


23 (3.2) 


560 (5.8) 


23 (3.6) 


554 (6.2) 

21 (3.1) 


533 (5.5) 

14 (0.8) 

Ontario, Canada 

■ 

1 

20 (2.9) 

549 (6.3) 

II 

31 (4.0) 

II 

550 (5.1) 

II 

33 (3.5) 

II 

553 (4.9) 

17(2.7) 

II 

551 (7.3) 

12 (0.5) 

Quebec, Canada 



33 (4.2) 

538 (3.7) 


40 (4.6) 


539 (3.2) 


20 (3.6) 


533 (6.7) 

7(1.9) 


537 (6.4) 

16 (0.7) 

Maltese - Malta 

r 

1 

17(0.1) 

467 (3.7) 

II 

35 (0.1) 

II 

456 (2.3) 

II 

34 (0.1) 

II 

454 (2.6) 

14 (0.1) 

II 

460 (3.7) 

12 (0.0) 

Eng/Afr (5) - RSA 



43 (5.0) 

415 (15.3) 


24 (4.6) 


420 (14.8) 


15 (3.5) 


450 (25.3) 

17 (4.7) 


455 (22.5) 

17(1.1) 

Andalusia, Spain 

■ 

1 

58 (3.9) 

521 (3.2) 

II 

16(3.1) 

II 

520 (5.7) 

II 

11 (2.6) 

II 

501 (8.7) 

15 (2.8) 

II 

498 (7.2) 

21 (1.0) 

Abu Dhabi, UAE 



15(3.2) 

397 (12.8) 


31 (4.4) 


428 (13.5) 


28 (4.1) 


419 (10.4) 

26(3.7) 


438 (9.7) 

10 (0.6) 

Dubai, UAE 

r 

1 

13 (2.6) 

492 (10.3) 

II 

28 (3.6) 

II 

477 (9.7) 

II 

38 (4.3) 

II 

482 (8.8) 

21 (2.6) 

II 

470 (8.5) 

11 (0.6) 

Florida, US 

r 


26 (4.3) 

572 (8.6) 


33 (4.3) 


576 (6.1) 


30 (3.9) 


566 (6.6) 

11 (3.5) 


563 (8.0) 

15 (0.9) 


0 Republic of South Africa (RSA) tested 5th grade students receiving instruction in English (ENG) or Afrikaans (AFR). 


( ) Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 

A tilde (~) Indicates insufficient data to report achievement 

An “r" indicates data are available for at least 70% but less than 85% of the students. An ‘s' indicates data are available for at least 50% but less than 70% of the students. 
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SOURCE: lEA's Progress in International Reading Literacy Study - PIRLS 2011 




The two file types are merged and the resulting merged file is then input to the JACKPV macro. 
The merging is done using the combination of identification variables IDCNTRY, IDTEACH, and 
IDLINK. The combination of values for these three variables is unique within the teacher 
background data files, but is repeated in the student-teacher linkage data files as many times as 
needed to link a teacher to all students in a classroom. After the files are merged, the JACKPV macro 
is invoked and the results can be printed. 

For this analysis, we will again use the data for all available countries, making use of an 
aggregated teacher background data file, ATGALLR3, and an aggregated student-teacher linkage 
data file, ASTALLR3. These aggregated files were created with the JOIN macro. 

The SAS program that executes this example is presented in Exhibit 3.20 and is included in the 
International Database under the name EXAMPLE5.SAS. Note that one of the steps in this program 
is to select only those students who have non-missing data in our variable of interest ATDG01. The 
results obtained from this program are displayed in Exhibit 3.21, edited to show only the first four 
countries for the sake of brevity. 

Exhibit 3.20: Example SAS Program for Teacher-level Analysis (EXAMPLE5.SAS) 


LIBNAME Pll " <datpath> " ; 

% INCLUDE "<macpath> JACKPV. SAS" ; 

PROC SORT DATA = Pll . ATGALLR3 OUT = ATGALLR3 ; 

BY IDCNTRY IDTEACH IDLINK ; 

PROC SORT DATA = Pll . ASTALLR3 OUT = ASTALLR3 ; 

BY IDCNTRY IDTEACH IDLINK ; 

DATA ATGMERGED ; 

MERGE ATGALLR3 (IN = INATG) 

ASTALLR3 (IN = INAST) ; 

BY IDCNTRY IDTEACH IDLINK ; 

IF INATG AND INAST ; 

DATA ATGMERGED ; 

SET ATGMERGED ; 

WHERE NMISS (ATDG01) = 0 ; 

PROC FORMAT LIBRARY = WORK ; 

VALUE COUNTRY 

< list PIRLS 2011 country formats > ; 

VALUE ATDG 

1 = "20 YEARS OR MORE" 

2 = "AT LEAST 10 BUT LESS THAN 20 YEARS" 

3 = "AT LEAST 5 BUT LESS THAN 10 YEARS" 

4 = "LESS THAN 5 YEARS" ; 

% JACKPV (MATWGT , JKZONE , JKREP, 75, IDCNTRY ATDG 01 , ASRREA0 , 5, ATGMERGED) ; 

PROC PRINT DATA = FINAL NOOBS ; 

VAR IDCNTRY ATDG 0 IN MATWGT MNPV MNPV_SE PCT PCT_SE ; 

FORMAT IDCNTRY COUNTRY. ATDG 01 ATDG. N 6.0 MATWGT 10.0 
MNPV MNPV_SE PCT PCT_SE 6.2 ; 


Each country’s results are displayed on four lines, one for each value of the ATDG01 variable. 
The results are presented in the same manner as in the previous examples, with countries identified 
in the first column, and the second column describing the categories of the analysis variable. As 
shown in the first four lines of results, 41.50 percent (standard error of 3.98) of fourth grade students 
in Australia were taught by teachers with 20 years or more of experience, 22.37 percent (standard 
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error of 3.52) by teachers with 10 to 20 years of experience, 19.03 percent (standard error of 2.88) by 
teachers with 5 to 9 years of experience, and 17.09 percent (standard error of 3.25) by teachers with 
less than 5 years of experience. Also, the estimated average reading achievement was 529.67 
(standard error of 5. 13) for fourth grade students taught by teachers with 20 years or more of 
experience, 533.03 (standard error of 5.68) for students taught by teachers with 10 to 20 years of 
experience, 529.09 (standard error of 6.32) for students taught by teachers with 5 to 9 years of 
experience, and 533.76 (standard error of 7.24) for students taught by teachers with less than 5 years 
of experience. 

Exhibit 3.21 : Output for Example Teacher-level Analysis (EXAMPLE 5) 


IDCNTRY 

ATDG01 




N 

TCHWGT 

MNPV 

MNPV_SE 

PCT 

PCT_SE 

AUSTRALIA 

20 YEARS OR MORE 




1954 

83004 

529.67 

5 . 13 

41.50 

3 . 98 

AUSTRALIA 

AT LEAST 10 BUT LESS 

THAN 

20 

YEARS 

1066 

44750 

533 . 03 

5.68 

22 .37 

3.52 

AUSTRALIA 

AT LEAST 5 BUT LESS 

THAN 

10 

YEARS 

1000 

38063 

529 . 09 

6.32 

19 . 03 

2 . 88 

AUSTRALIA 

LESS THAN 5 YEARS 




868 

34188 

533 . 76 

7.24 

17 . 09 

3.25 

AUSTRIA 

20 YEARS OR MORE 




2445 

42361 

532.39 

2 .42 

54 . 79 

2 . 83 

AUSTRIA 

AT LEAST 10 BUT LESS 

THAN 

20 

YEARS 

1161 

19233 

525 . 88 

4 . 04 

24 . 87 

2 . 73 

AUSTRIA 

AT LEAST 5 BUT LESS 

THAN 

10 

YEARS 

523 

8339 

532.30 

3 . 93 

10 . 78 

1 . 93 

AUSTRIA 

LESS THAN 5 YEARS 




558 

7389 

512.69 

6 . 76 

9.56 

1 . 88 

AZERBAIJAN 

20 YEARS OR MORE 




2490 

53345 

466.47 

3 . 77 

59 . 79 

4.51 

AZERBAIJAN 

AT LEAST 10 BUT LESS 

THAN 

20 

YEARS 

1139 

23144 

456 . 15 

8.56 

25 . 94 

3 . 11 

AZERBAIJAN 

AT LEAST 5 BUT LESS 

THAN 

10 

YEARS 

349 

9506 

439 . 81 

10.43 

10.65 

2.68 

AZERBAIJAN 

LESS THAN 5 YEARS 




121 

3231 

483 . 72 

33.44 

3.62 

1 . 95 

BELGIUM (FRENCH) 

20 YEARS OR MORE 




1445 

19025 

516 . 19 

3 . 91 

40.39 

3 . 88 

BELGIUM (FRENCH) 

AT LEAST 10 BUT LESS 

THAN 

20 

YEARS 

1085 

15154 

501 . 83 

4 . 95 

32 . 17 

3 . 83 

BELGIUM (FRENCH) 

AT LEAST 5 BUT LESS 

THAN 

10 

YEARS 

534 

7413 

503 . 82 

6 . 72 

15 . 74 

3.45 

BELGIUM (FRENCH) 

LESS THAN 5 YEARS 




431 

5507 

498.39 

12.47 

11 . 69 

2.53 


3.1 0 PIRLS/prePIRLS Analyses with School-level Variables 

Because PIRLS 2011 has representative samples of schools, it is possible to compute reasonable 
statistics with schools as units of analysis. However, the school samples were designed to optimize 
the student samples and the student-level estimates. For this reason, it is preferable to analyze 
school-level variables as attributes of students, rather than as elements in their own right. Therefore, 
analyzing school data should be done by linking the students to their schools. 

In general, to perform analyses using the school background data fdes, users should do the 
following: 

1 . Identify the variables of interest in the school and student background data files and note any 
specific national adaptations to the variables. 

2. Retrieve the relevant variables from the school background data files, including analysis 
variables, classification variables, identification variables (IDCNTRY and IDSCHOOL), and any 
other variables used in the selection of cases. 
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3. Retrieve the relevant variables from the student background data files, including plausible values 
of achievement, classification variables, identification variables (IDCNTRY and IDSCHOOL), 
sampling (JKZONE and JKREP) and weighting (TOTWGT) variables, and any other variables 
used in the selection of cases. 

4. Merge the school background data files with the student background data files using the variables 
IDCNTRY and IDSCHOOL. 

5. Perform any necessary variable transformations or recodes. 

6. Use the macros JACKGEN and JACKREG, or JACKPV and JACKREGP if plausible values are 
involved, with the appropriate parameters. 

7. Specify the location of the data files (<datpath>) and the macros (<macpath>). 

8. Print the results file. 

Our example of an analysis using school background data will compute the percentages of fourth 
grade students, with their average reading achievement, who attended schools composed of students 
with different levels of economic background. The results of this analysis are presented in 
Exhibit 5.2 of PIRLS 2011 International Results in Reading and are replicated here in Exhibit 3.22. 

The information for this analysis is found in the school-level derived variable ACDG03, where 
schools are characterized as being composed of more affluent students than disadvantaged students, 
composed of more disadvantaged students than affluent students, or composed of neither more 
affluent students nor more disadvantaged students. 5 

Because we are using a school-level variable, we need to use the school background data files 
and the student background data files to find the variables. From the school background data files, 
we need the variable that contains the information on the schools’ composition of students based on 
their economic background (ACDG03) and the identification variables IDCNTRY and IDSCHOOL 
that will allow us to link the school data to the student data. 

Next, we retrieve the variables of interest from the student background data files. We need the 
country and school identification variables (IDCNTRY and IDSCHOOL) necessary to merge the 
school data to the student data. We also need the jackknife replication variables (JKZONE and 
JKREP), the student weighting variable (TOTWGT), and the fourth grade reading achievement 
plausible values (ASRREA01 through ASRREA05). 

We then proceed to merge the school data with the student data using the variables IDCNTRY 
and IDSCHOOL and use the macro JACKPV to obtain the percentages of students, with their 
average achievement, in each category of the variable ACDG03 for each country. For this analysis, 
we will use the data for all available countries, making use of an aggregated school file ACGALLR3 
and an aggregated student file ASGALLR3. These aggregated files were created with the JOIN 
macro. 


5 


Supplement 3 to this User Guide describes the construction of the school-level derived variable ACDG03 from responses to questions posed to the 
school principals. 
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Exhibit 3.22: Exhibit of Example School-level Analysis, Taken from 
PIRLS 2011 International Results in Reading (Exhibit 5.2) 


Exhibit 5.2: School Composition by Student Economic Background 

Reported by Principals 


Country 


More Affluent - Schools Where More 
than 25% of Students Come from 
Economically Affluent Homes and Not 
More than 25% from Economically 
Disadvantaged Homes 


Neither More Affluent nor 
More Disadvantaged 


PIRLS 20111H 

More Disadvantaged - Schools Where 
More than 25% of Students Come 
from Economically Disadvantaged 
Homes and Not More than 25% from 
Economically Affluent Homes 




Percent 


Average 

Percent 


Average 


Percent 


Average 




of Students 


Achievement 

of Students 


Achievement 


of Students 


Achievement 


Australia 


32(3.9) 


556 (3.9) 

41 (4.0) 


526 (3.6) 


27 (3.4) 


500 (5.7) 


Austria 

■■ 

31 (4.0) 

■I 

539 (2.9) 

48 (3.8) 

■■ 

532 (2.4) 

■■ 

21 (3.9) 

■■ 

505 (4.4) 

■ 

Azerbaijan 

r 

11 (2.5) 


464 (10.5) 

32(4.7) 


470 (8.1) 


57 (4.9) 


462 (6.1) 


Belgium (French) 


49 (5.5) 

■1 

522 (3.1) 

27(4.5) 

■■ 

501 (6.4) 

■■ 

25(4.5) 

■■ 

483 (4.6) 

■ 

Bulgaria 


15(3.3) 


561 (6.2) 

43 (4.3) 


549 (4.5) 


42 (4.4) 


506 (7.8) 


Canada 

■■ 

39 (2.4) 

■I 

557 (3.0) 

34 (2.9) 

■■ 

549 (2.7) 

■■ 

28 (2.6) 

■■ 

533 (2.6) 

■ 

Chinese Taipei 


22(3.3) 


563 (4.5) 

67(3.5) 


554 (2.4) 


11 (2.0) 


525 (6.8) 


Colombia 

■■ 

7(2.0) 

■1 

521 (17.1) 

15 (3.4) 

■■ 

471 (11.7) 

■■ 

78 (3.9) 

■I 

432 (4.7) 

■ 

Croatia 


38 (4.0) 


560 (3.1) 

38 (4.2) 


550 (2.3) 


24(3.2) 


551 (4.9) 


Czech Republic 


37 (3.7) 


551 (3.2) 

46 (4.4) 

■■ 

548 (2.2) 

■■ 

17(3.1) 

■I 

524 (6.7) 

■ 

Denmark 


60 (3.6) 


561 (2.1) 

33 (3.3) 


546 (3.0) 


7(1.8) 


524 (7.7) 


England 

■■ 

32 (4.8) 

■■ 

568 (4.9) 

33 (4.9) 

■I 

554 (4.0) 

■■ 

35 (4.0) 

■I 

527 (4.7) 

■ 

Finland 


43 (4.2) 


576 (2.4) 

47 (4.3) 


567 (2.7) 


10 (2.6) 


541 (4.0) 


France 

■■ 

37(4.3) 

■I 

539(3.5) 

35 (3.9) 

■■ 

522 (4.6) 

■■ 

28(3.7) 

■■ 

493 (4.7) 

■ 

Georgia 


16 (3.0) 


496 (8.8) 

41 (4.3) 


494 (5.9) 


43 (4.0) 


480 (4.5) 


Germany 

■■ 

21 (2.8) 

■I 

555 (3.3) 

53 (3.7) 

■I 

549 (3.0) 

■■ 

26(3.3) 

■■ 

512(5.5) 

■ 

Hong Kong SAR 


20(3.3) 


580 (3.9) 

30 (4.7) 


569 (5.3) 


50 (4.7) 


568 (4.4) 


Hungary 


21 (3.6) 

■I 

573 (6.3) 

31 (4.3) 

■■ 

557 (4.2) 

■■ 

48 (4.0) 

■■ 

516 (5.2) 

■ 

Indonesia 

r 

20 (4.1) 


475 (5.6) 

21 (3.9) 


431 (7.7) 


59 (4.6) 


421 (6.0) 


Iran, Islamic Rep. of 

■■ 

27 (3.6) 

■I 

488 (7.6) 

27 (4.1) 

■■ 

460 (6.7) 

■■ 

46 (4.2) 

■■ 

438 (4.8) 

■ 

Ireland 

r 

39(4.7) 


568 (3.3) 

30 (4.2) 


554 (4.7) 


31 (3.4) 


523 (4.0) 


Israel 

■■ 

35 (3.6) 

■I 

566 (6.1) 

28 (3.4) 

■■ 

559 (4.9) 

■■ 

37 (3.4) 

■I 

500 (6.5) 

■ 

Italy 


37(3.8) 


541 (4.5) 

43 (3.7) 


545 (3.6) 


20 (2.9) 


531 (5.0) 


Lithuania 

■■ 

19 (3.3) 

■I 

552 (5.8) 

43 (4.6) 

■■ 

529 (33) 

■■ 

38(3.5) 

■■ 

518 (3.1) 

■ 

Malta 


47 (0.1) 


482 (2.2) 

43 (0.1) 


478 (2.4) 


10 (0.1) 


421 (5.3) 


Morocco 

■■ 

12(2.1) 

■I 

372 (16.7) 

13 (2.8) 

■■ 

317 (11.6) 

■■ 

75(3.3) 

■■ 

304 (6.1) 

■ 

Netherlands 

r 

63 (4.9) 


553 (2.3) 

23 (3.9) 


544 (2.6) 


15 (3.8) 


522 (8.2) 


New Zealand 

■■ 

39 (3.4) 

■I 

560 (3.2) 

34 (3.6) 

■■ 

533 (3.7) 

■■ 

27(2.5) 

■■ 

489 (4.2) 

■ 

Northern Ireland 

r 

36(4.7) 


578 (4.9) 

38 (4.3) 


555 (3.3) 


26 (3.8) 


534 (5.8) 


Norway 

■■ 

53(5.3) 

■I 

511 (3.3) 

44 (5.3) 

■■ 

505 (2.9) 

■■ 

3(1.2) 

■■ 

488 (16.9) 

■ 

Oman 

r 

44 (3.4) 


3% (4.3) 

25 (2.9) 


378 (6.7) 


31 (2.9) 


370 (5.1) 


Poland 

■■ 

8(2.1) 

■I 

536 (10.2) 

61 (3.8) 

■■ 

532 (2.9) 

■■ 

31 (3.7) 

■■ 

512 (3.5) 

■ 

Portugal 


30 (4.5) 


552 (4.0) 

39 (4.9) 


547 (4.1) 


31 (4.8) 


522 (4.6) 


Qatar 

■■ 

68 (3.0) 

■I 

423 (5.0) 

21 (2.3) 

■I 

441 (10.6) 

■■ 

11 (1.9) 

■I 

378 (7.5) 

■ 

Romania 


19(3.1) 


538 (9.2) 

24 (4.0) 


507 (8.8) 


57(4.8) 


491 (6.5) 


Russian Federation 

■■ 

58 (3.2) 

■I 

576 (4.0) 

29(3.3) 

■■ 

562 (4.8) 

■■ 

13 (2.1) 

■I 

549 (8.5) 

■ 

Saudi Arabia 

r 

42 (4.7) 


445 (8.5) 

30 (4.3) 


439 (6.0) 


29 (4.0) 


408 (10.0) 


Singapore 


40 (0.0) 


590 (5.2) 

50 (0.0) 

■■ 

556 (4.5) 

■■ 

10 (0.0) 

■I 

541 (14.3) 

■ 

Slovak Republic 


24(3.3) 


551 (3.9) 

56 (3.4) 


542 (2.5) 


20 (3.2) 


499 (8.0) 


Slovenia 

■■ 

42 (4.0) 

■I 

533 (3.7) 

40 (4.0) 

■■ 

531 (2.7) 

■■ 

18 (3.0) 

■■ 

521 (6.7) 

■ 

Spain 


51 (3.7) 


527 (4.0) 

31 (3.3) 


511 (4.7) 


18 (3.1) 


482 (5.1) 


Sweden 

■■ 

76(4.2) 

■I 

547 (2.8) 

17 (4.1) 

■■ 

532 (73) 

■■ 

7(1.5) 

■■ 

509 (8.5) 

■ 

Trinidad and Tobago 


20(3.2) 


508 (75) 

26 (4.0) 


464 (9.2) 


54 (4.2) 


460 (5.7) 


United Arab Emirates 

r 

68 (2.2) 

■I 

440 (3.6) 

20 (1.6) 

■■ 

444 (6.2) 

■■ 

12(1.7) 

■■ 

412 (5.5) 

■ 

United States 

r 

18(2.2) 


591 (2.9) 

31 (2.6) 


570 (3.5) 


51 (2.3) 


537 (2.4) 


International Avg. 


35 (0.5) 


530 (0.9) 

35 (0.6) 


515 (0.8) 


30 (0.5) 


490 (1.0) 


Sixth Grade Participants 

Botswana 


32(3.6) 


454 (7.9) 

25 (4.0) 


403 (8.3) 


43 (4.3) 


390 (4.3) 


Honduras 

■■ 

16 (4.0) 

■1 

518 (14.5) 

13 (3.8) 

■■ 

440 (14.3) 

■■ 

71 (4.9) 

■■ 

444 (5.7) 

■ 

Kuwait 

r 

30 (4.5) 


429 (13.9) 

35 (4.9) 


431 (14.1) 


35(5.3) 


402 (11.2) 


Morocco 

■■ 

12(2.3) 

■I 

465 (16.5) 

12 (2.6) 

■■ 

456 (11.1) 

■■ 

76(3.1) 

■■ 

415 (5.8) 

■ 

Benchmarking Participants 0 

Alberta, Canada 


35 (3.7) 


564 (4.8) 

40 (4.3) 


550 (4.3) 


25(3.8) 


527(5.5) 


Ontario, Canada 

■■ 

32(4.7) 

■1 

557 (5.4) 

34(5.3) 

■■ 

555 (4.3) 

■■ 

34 (5.4) 

■■ 

537 (4.4) 

■ 

Quebec, Canada 


60 (4.1) 


544 (2.6) 

25 (4.0) 


526 (5.2) 


15(2.7) 


528 (4.9) 


Maltese - Malta 

■■ 

47 (0.1) 

■1 

459 (2.1) 

43 (0.2) 

■■ 

467 (2.6) 

■■ 

10 (0.1) 

■■ 

419 (4.5) 

■ 

Eng/Afr (5) - RSA 

r 

22 (4.1) 


507 (15.1) 

23 (6.1) 


419 (20.4) 


55 (6.7) 


382 (13.5) 


Andalusia, Spain 

■■ 

47 (4.3) 

■1 

525 (3.9) 

34 (3.5) 

■■ 

519 (3.2) 

■■ 

19 (3.7) 

■■ 

490 (5.9) 

■ 

Abu Dhabi, UAE 

s 

75 (4.5) 


423 (7.4) 

12(3.2) 


422 (18.7) 


13 (3.5) 


402 (10.7) 


Dubai, UAE 

■■ 

67 (0.4) 

■1 

473 (2.8) 

22 (0.3) 

■■ 

498 (4.3) 

■■ 

11 (0.2) 

■■ 

416 (5.0) 

■ 

Florida, US 

r 

11 (4.6) 


598 (5.1) 

20 (4.7) 


590 (9.1) 


68 (4.7) 


559 (4.5) 



0 Republic of South Africa (RSA) tested 5th grade students receiving instruction in English (ENG) or Afrikaans (AFR). 

( ) Standard errors appear In parentheses. Because of rounding some results may appear Inconsistent. 

An "r" indicates data are available for at least 70% but less than 85% of the students. An 's' indicates data are available for at least 50% but less than 70% of the students. 
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SOURCE: lEA's Progress in International Reading Literacy Study - PIRLS 201 1 



The SAS program that implements this example is presented in Exhibit 3.23 and is included in 
the International Database under the name EXAMPLE6.SAS. The results of this program are 
displayed in Exhibit 3.24, edited to show only the first four countries for the sake of brevity. Note 
that one of the steps in this program is to select only those students who have non-missing data in our 
variable of interest ACDG03. 

Exhibit 3.23: Example SAS Program for School-level Analysis (EXAMPLE6.SAS) 


LIBNAME Pll " <datpath> " ; 

% INCLUDE " <macpath> JACKPV . SAS " ; 

PROC SORT DATA = Pll . ACGALLR3 OUT = ACGALLR3 ; 
BY IDCNTRY IDSCHOOL ; 

PROC SORT DATA = Pll . ASGALLR3 OUT = ASGALLR3 ; 
BY IDCNTRY IDSCHOOL ; 

DATA ACGMERGED ; 

MERGE ACGALLR3 (IN = INACG) 

ASGALLR3 (IN = INASG) ; 

BY IDCNTRY IDSCHOOL ; 

IF INACG AND INASG ; 

DATA ACGMERGED ; 

SET ACGMERGED ; 

WHERE NMISS (ACDG03 ) = 0 ; 

PROC FORMAT LIBRARY = WORK ; 


VALUE COUNTRY 

< list 2011 country formats > 

VALUE ACDG 

1 = "MORE AFFLUENT LESS DISADVANTAGED" 

2 = "NEITHER" 

3 = "MORE DISADVANTAGED LESS AFFLUENT" ; 

% JACKPV (TOTWGT , JKZONE , JKREP , 75, IDCNTRY ACDG03 , ASRREAO , 5, ACGMERGED) ; 

PROC PRINT DATA = FINAL NOOBS ; 

VAR IDCNTRY ACDG03N TOTWGT MNPV MNPV_SE PCT PCT_SE ; 

FORMAT IDCNTRY COUNTRY. ACDG 03 ACDG. N 6.0 TOTWGT 10.0 
MNPV MNPV_SE PCT PCT_SE 6.2 ; 


Exhibit 3.24: Output for Example School-level Analysis (EXAMPLE 6) 


IDCNTRY 

ACDG03 


N 

TOTWGT 

MNPV 

MNPV_SE 

PCT 

PCT_SE 

AUSTRALIA 

MORE AFFLUENT LESS 

DISADVANTAGED 

1759 

71259 

555 . 68 

3 . 94 

32 . 08 

3 . 91 

AUSTRALIA 

NEITHER 


2425 

90918 

526 . 01 

3.57 

40 . 93 

3 . 99 

AUSTRALIA 

MORE DISADVANTAGED 

LESS AFFLUENT 

1162 

59962 

499.56 

5 . 70 

26 . 99 

3.42 

AUSTRIA 

MORE AFFLUENT LESS 

DISADVANTAGED 

1299 

21774 

539.43 

2 . 91 

30.53 

4 . 04 

AUSTRIA 

NEITHER 


1961 

34354 

531.63 

2 .40 

48 . 16 

3 . 79 

AUSTRIA 

MORE DISADVANTAGED 

LESS AFFLUENT 

1041 

15199 

505.26 

4.44 

21.31 

3 . 91 

AZERBAIJAN 

MORE AFFLUENT LESS 

DISADVANTAGED 

458 

9540 

463.52 

10.46 

11 . 34 

2.50 

AZERBAIJAN 

NEITHER 


1271 

26743 

470 . 17 

8 . 10 

31 . 80 

4 . 71 

AZERBAIJAN 

MORE DISADVANTAGED 

LESS AFFLUENT 

2368 

47824 

462 . 14 

6 . 10 

56 . 86 

4 . 88 

BELGIUM (FRENCH) 

MORE AFFLUENT LESS 

DISADVANTAGED 

1558 

20853 

522 .40 

3 . 06 

48.55 

5.48 

BELGIUM (FRENCH) 

NEITHER 


871 

11519 

501 . 01 

6.37 

26 . 82 

4.47 

BELGIUM (FRENCH) 

MORE DISADVANTAGED 

LESS AFFLUENT 

743 

10577 

483.24 

4.60 

24.63 

4.49 


In Exhibit 3.24, each country’s results are presented on three lines, one for each value of the 
ACDG03 variable. The results are presented much in the same manner as in previous examples, 
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where the first column identifies the countries and the second column describes the category of 
ACDG03 being reported. 

As shown in the first three lines of results, 32.08 percent of fourth grade students in Australia 
attended schools with more affluent students than disadvantaged students (standard error of 3.91), 
40.93 percent attended schools with neither more affluent students nor more disadvantaged students 
(standard error of 3.99), and 26.99 percent attended schools with more disadvantaged students than 
affluent students (standard error of 3.42). Also, the estimated average reading achievement 
was 555.68 (standard error of 3.94) for students in schools with more affluent students, 526.01 
(standard error of 3.57) for students in schools with neither more affluent students nor more 
disadvantaged students, and it was 499.56 (standard error of 5.70) for students in schools with more 
disadvantaged students. 
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Chapter 4 

The PIRLS 2011 
International Database Files 


4.1 Overview 

The PIRLS 2011 International Database contains achievement data and student, home, teacher, and 
school background data collected in the 48 countries and 9 benchmarking participants that took part 
in PIRLS 201 1 and prePIRLS 2011. Exhibit 4.1 lists all of the PIRLS and prePIRLS 201 1 countries 
and benchmarking participants, along with identifying codes used in the International Database. The 
database also contains materials that provide additional information on its structure and contents. 
This chapter describes the contents of the database with a special emphasis on the actual data files 
used in Chapters 2 and 3 of this User Guide to analyze the PIRLS 201 1 assessment results. The next 
section introduces this User Guide and its supplements. The following section describes the item- 
related materials included in the international database. The final four sections present the different 
data files and related materials included in the database, as described in Exhibit 1 . 1 of Chapter 1 . 


4.2 The PIRLS 201 1 User Guide 


The PIRLS 2011 User Guide is an important resource to the PIRLS 2011 International Database 
because it provides a detailed description of its structure and contents, as well as instructions to make 
the best use of its data. The User Guide also includes a series of four supplements in printable PDF 
format: 


• Supplement 1 : International Version of the PIRLS 2011 Background Questionnaires and 
Curriculum Questionnaire — Supplement 1 includes the international version of all background 
questionnaires and the curriculum questionnaire administered in PIRLS and prePIRLS 201 1. 1 It 
is a good reference guide to understand what questions were asked and the variable names under 
which the responses are recorded in the International Database. 

• Supplement 2: National Adaptations of International Background Questionnaires — Supplement 2 
provides details on all national adaptations that were applied to the national version of all 
PIRLS 2011 background questionnaires. Users should refer to this supplement for any special 
national adaptations to background variables that could potentially affect the results of analyses. 


1 The prePIRLS 2011 participants administered the same background questionnaires as the PIRLS 2011 participants. 
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Exhibit 4.1 : Countries Participating in PIRLS and prePIRLS 201 1 


Country 

ISO Code 

Assessment 

Alpha 

Numeric 

2011 

2006 

2001 

Australia 

AUS 

036 

• 



Austria 

AUT 

040 

• 

• 


Azerbaijan 

AZE 

031 

• 



Belgium (French) 

BFR 

957 

• 

• 


Bulgaria 

BGR 

100 

• 

• 

• 

Canada 

CAN 

124 

• 



Chinese Taipei 

TWN 

158 

• 

• 


Colombia 

COL 

170 

• 


• 

Croatia 

HRV 

191 

• 



Czech Republic 

CZE 

203 

• 


• 

Denmark 

DNK 

208 

• 

• 


England 

ENG 

926 

• 

• 

• 

Finland 

FIN 

246 

• 



France 

FRA 

250 

• 

• 

• 

Georgia 

GEO 

268 

• 

• 


Germany 

DEU 

276 

• 

• 

• 

Hong KongSAR 

HKG 

344 

• 

• 

• 

Hungary 

HUN 

348 

• 

• 

• 

Indonesia 

IDN 

360 

• 

• 


Iran, Islamic Rep. of 

IRN 

364 

• 

• 

• 

Ireland 

IRL 

372 

• 



Israel 

ISR 

376 

• 

O 

o 

Italy 

ITA 

380 

• 

• 

• 

Kuwait 

KWT 

414 


• 

• 

Lithuania 

LTU 

440 

• 

• 

• 

Malta 

MLT 

470 

• 



Morocco 

MAR 

504 

• 

o 

o 

Netherlands 

NLD 

528 

• 

• 

• 

New Zealand 

NZL 

554 

• 

• 

• 

Northern Ireland 

NIR 

928 

• 



Norway 

NOR 

578 

• 

• 

• 

Oman 

OMN 

512 

• 



Poland 

POL 

616 

• 

• 


Portugal 

PRT 

620 

• 



Qatar 

QAT 

634 

• 

o 

o 

Romania 

ROM 

642 

• 

• 

• 

Russian Federation 

RUS 

643 

• 

• 

• 

Saudi Arabia 

SAU 

682 

• 



Singapore 

SGP 

702 

• 

• 

• 

Slovak Republic 

SVK 

703 

• 

• 

• 

Slovenia 

SVN 

705 

• 

• 

• 

South Africa 

ZAF 

710 


• 


Spain 

ESP 

724 

• 

• 
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Exhibit 4.1 : Countries Participating in PIRLS and prePIRLS 201 1 (Continued) 


ISO Code 

Assessment 

LUUMliy 

Alpha 

Numeric 

2011 

2006 

2001 

Sweden 

SWE 

752 

• 

• 

• 

Trinidad and Tobago 

TTO 

780 

• 

• 


United Arab Emirates 

ARE 

784 

• 



United States 

USA 

840 

• 

• 

• 

Sixth Grade Countries 

Botswana 

BWA 

072 

• 



Honduras 

HND 

340 

• 



Kuwait 

KWT 

414 

• 



Morocco 

MA6 

6504 

• 



Benchmarking Participants 

Alberta, Canada 

CAB 

9134 

• 

• 


Ontario, Canada 

COT 

9132 

• 

• 

• 

Quebec, Canada 

CQU 

9133 

• 

• 

• 

Maltese -Malta 

MLN 

9470 

• 



Eng/Afr (5) - RSA 

ZAF 

710 

• 

• 


Andalusia, Spain 

EAN 

72401 

• 



Abu Dhabi, UAE 

AAD 

7842 

• 



Dubai, UAE 

ADU 

7841 

• 



Florida, US 

UFL 

11200 

• 



prePIRLS Countries 





Botswana 

BWA 

072 

• 


Colombia 

COL 

170 

• 


South Africa 

ZAF 

710 

• 


• Indicates participation in a testing cycle. 


° Indicates participation but data not 
comparable for measuring trends to 201 1 . 


• Supplement 3: Variables Derived from the Student, Home, Teacher, and School Questionnaire 
Data — Supplement 3 describes how the derived background variables were computed to produce 
exhibits in PIRLS 2011 International Results in Reading (Mullis, Martin, Foy, & Drucker, 2012). 

• Supplement 4: PIRLS 201 1 Sampling Stratification Information — Supplement 4 provides the 
labels assigned to the national explicit and implicit strata defined during the sampling process. 

4.3 PIRLS/prePIRLS 201 1 Achievement Items Documentation 

A number of documents related to the PIRLS and prePIRLS 2011 achievement items are available 
for download along with the International Database. They include detailed information on the PIRLS 
and prePIRLS 201 1 released passages and items, summary information on all PIRLS and prePIRLS 
2011 items, as well as the IRT item parameters estimated from the PIRLS 2011 concurrent item 
calibration and the prePIRLS 2011 scaling. 
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The International Database includes PDF documents of the PIRLS and prePIRLS 2011 released 
passages and achievement items. The documents include the items themselves with descriptive 
information and the scoring guides for the constructed response items. The International Database 
also includes percent correct statistics for all released items in Excel and PDF format. There are 
separate files for PIRLS and prePIRLS. 

Achievement item information files are provided to enable users of the PIRLS 2011 International 
Database to readily produce summaries of item characteristics. There are separate achievement item 
information files for PIRLS and prePIRLS. These files are available in Excel format and include the 
following information for each item in the PIRLS and prePIRLS 2011 assessments: 

• The item’s permanent and unique identifier; 

• The item’s passage name and its sequential location; 

• The item’s label; 

• The item’s purpose for reading and process of comprehension; 

• The item’s type, either multiple choice or constructed response; 

• The number of options for a multiple choice item; 

• The correct response key for a multiple choice item; 

• The item’s point value; 

• An indicator showing if the item was included in the IRT scaling; and 

• An indicator showing if the item was released after the 201 1 assessment. 

The International Database also includes Excel files with the IRT item parameters estimated for 
all 2006 and 2011 items from the PIRLS 2011 concurrent item calibrations, as well as for the 
prePIRLS 201 1 scaling. These same item parameters are presented in the scaling section of Methods 
and Procedures in TIMSS and PIRLS 2011 (Martin & Mullis, 2012). Finally, the database also 
includes Excel tables with the scale transformations that were applied to the PIRLS and prePIRLS 
2011 reading scales. 

4.4 The PIRLS/prePIRLS 201 1 Data Files 

The PIRLS 201 1 International Database includes the actual data from all instruments administered to 
the students, their parents, their teachers, and their school principals for both PIRLS 2011 and 
prePIRLS 2011. This includes the student responses to the achievement items and the responses to 
the student, home, teacher, and school background questionnaires. These data files also include the 
achievement scores estimated for participating students as well as background variables derived for 
reporting in PIRLS 2011 International Results in Reading. National Research Coordinators’ 
responses to the curriculum questionnaire also are part of the International Database and are 
described later in this chapter. 

This section describes the contents and format of the PIRLS and prePIRLS 2011 data files. 

With the exception of the curriculum data file, the PIRLS data files are provided in SAS 
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(SAS Institute, 2011) export format (.EXP) and SPSS (IBM Corporation, 2012) format (.SAV). 
Data files are provided for each country that participated in PIRLS and prePIRLS 2011 and for 
which internationally comparable data are available. The file names given to the various data file 
types are shown in Exhibit 4.2. For example, ASGNORR3.SAV is an SPSS file that contains 
Norway’s PIRLS 2011 student background data. For each file type, a separate data file is provided 
for each participating country. All data files and the variables they contain are described in the 
following sections. 

Exhibit 4.2: PIRLS/prePIRLS 201 1 Data File Names 


File Names 

Descriptions 

ACG***R3 

PIRLS 2011 school background data files 

ASA*** R3 

PIRLS 2011 student achievement data files 

ASG*** R3 

PIRLS 2011 student background data files 

ASH### R3 

PIRLS 2011 home background data files 

ASR*** R3 

PIRLS 2011 within-country scoring reliability data files 

AST### R3 

PIRLS 2011 student-teacher linkage files 

ATG### R3 

PIRLS 2011 teacher background data files 

ACG###L1 

prePIRLS 2011 school background data files 

ASA### LI 

prePIRLS 2011 student achievement data files 

ASG### LI 

prePIRLS 2011 student background data files 

ASH### LI 

prePIRLS 2011 home background data files 

ASR### LI 

prePIRLS 2011 within-country scoring reliability data files 

AST### LI 

prePIRLS 2011 student-teacher linkage files 

ATG### LI 

prePIRLS 2011 teacher background data files 


••• = 3-character country abbreviation based on the ISO 3166 coding scheme 
(see Exhibit 4.1). 


4.4.1 PIRLS/prePIRLS 201 1 Student Achievement Data Files (ASA) 

The PIRLS and prePIRLS 2011 student achievement data files contain the student responses to the 
individual achievement items in the PIRLS and prePIRLS 201 1 assessments. The student 
achievement data files are best suited for performing item-level analyses. Achievement scores 
(plausible values) for all of the PIRLS and prePIRLS 2011 achievement scales are available in the 
student achievement data files, as well as in the student background data files and student-teacher 
linkage data files. 

Students who participated in PIRLS 201 1 were administered one of 13 booklets and students in 
prePIRLS 2011 were administered one of 12 assessment booklets, each with two reading passages 
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and their accompanying items. 2 Some of these items were multiple choice items and some were 
constructed response items. The student achievement data files contain the actual responses to the 
multiple choice questions and the codes assigned to the constructed response items through the 
scoring guides. 

Item Variable Naming Convention 

The achievement item variable names are based on an 8-character alphanumeric code (e.g., 
R31M17CA), which adheres to the following rules: 

• The first character is either “R” for PIRLS items, or “L” for prePIRLS items. 

• The second character indicates the assessment cycle when the item was first used in PIRLS or 
prePIRLS. The items in the PIRLS 2011 assessment have either “1” for items produced in 2001, 
“2” for items produced in 2006, or “3” for new items in 201 1. All prePIRLS items have the code 
“1” because 2011 was its first administration. 

• The third character is always “1,” to denote the fourth grade. 

• The fourth character indicates the passage to which the item belongs. The following letters were 
used for PIRLS 2011: 

E — Fly, Eagle, Fly; 

F — Flowers on the Roof; 

G — The Giant Tooth Mystery; 

K — Sharks; 

L — Leonardo da Vinci; 

M — Empty Pot; 

N — Day Hiking; 

P — Enemy Pie; 

W — Where’s the Honey; and 
Y — Shiny Straw. 

• The following letters were used for prePIRLS 2011: 

A — Ants; 

B — Brave Charlotte; 

C — Caterpillar to Butterfly; 

D — Two Giant Dinosaurs; 

L — Lonely Giraffe; 

O — Baghita’s Perfect Orange; 

P — Training a Deaf Polar Bear; and 
U — Summer My Father Was Ten. 


2 The PIRLS and prePIRLS 2011 booklet design is described in Chapter 4 of the PIRLS 2011 Assessment Framework (Mullis, Martin, Kennedy, 
Trong, & Sainsbury, 2009). 
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• The fifth and sixth characters are a two-digit sequential number that identifies the sequential 
position of the item within the passage. 

• The seventh character indicates the item type. The letter “C” is used for constructed response 
items and the letter “M” for multiple choice items. 

• The eighth character indicates the item part, and appears only when required. Generally, it is a 
letter from “A” to “E,” depending how many parts there are to a particular item. The letter “Z” is 
used to represent a derived item where the scores of its item parts are combined into a single 
score for the derived item. 

For example, R31M17CA is the first part of a constructed response PIRLS reading item 
produced in 201 1 and that is the 17 th item in the passage “Empty Pot.” 

Item Response Code Values 

A series of conventions were adopted to code the data included in the PIRLS and prePIRLS data 
files. This section describes these conventions for the achievement items. 

The values assigned to each of the achievement item variables depend on the item format. For 
multiple choice items, numerical values from 1 through 4 were used to correspond to the response 
options A through D, respectively. For these items, the correct response code is included as part of 
the variable label in the achievement codebook files, and SAS and SPSS programs are included as 
part of the International Database to score these items. 3 For the constructed response items, 
numerical values ranging from 0 to 3 are used to correspond to the score points obtained from the 
students’ responses. 

For some items, students were asked to provide more than one answer, each one being scored 
separately. The pattern of responses across these item parts determined the score on the item as a 
whole. Generally, students provided answers to either two or three parts and the score on the item 
was the number of correct responses. For all of these items, each item part was worth one point, and 
the derived items were given the code values shown in Exhibit 4.3. The total score for a multi-part 
item is contained in a derived variable, indicated by a final character of Z in its item variable name. 
For example, derived variable R31M17CZ contains the combined score for its three parts labeled 
R31M17CA, R31M17CB, andR31M17CC. 

prePIRLS included a special four-part item — LI 1 A13CZ — whereby a student was given two 
points (full credit) if all four parts were answered correctly, only one point (partial credit) if only 
three parts were answered correctly, and zero points otherwise. 


3 The SPSS version of these programs function much like their SAS counterparts, described in Section 3.4 of Chapter 3. 
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Exhibit 4.3: 

PIRLS/prePIRLS 201 1 Derived Item Variable Codes 

Codes 

Descriptions 

For 3-point Derived Items: 

3 

Full credit (all three parts answered correctly) 

2 

Partial credit (only two parts answered correctly) 

1 

Partial credit (only one part answered correctly) 

0 

No credit (no points on any of the parts) 

For 2-point Derived Items: 

2 

Full credit (both parts answered correctly) 

1 

Partial credit (only one part answered correctly) 

0 

No credit (no points on any of the parts) 


Codes for Missing Values 

A subset of the values for each variable type was reserved for specific codes related to different 
categories of missing data. We recommend that the user read the following section with particular 
care, because the way in which these missing codes are used may have major consequences for 
analyses. 

Omitted Response Codes (SAS: . ; SPSS: 9) 

“Omitted” response codes were used for items that a student should have answered but did not. 

An omitted response code was given when an item was left blank or when two or more response 
options were checked for a multiple choice item. 

Not Administered Response Codes (SAS: .A ; SPSS: sysmis) 

Special codes were given to items that were “Not Administered” to distinguish these cases from data 
that were missing due to non-response. In general, the “Not Administered” code was used when an 
item was not administered, either by design arising from the rotation of items across the assessment 
booklets, or unintentionally when an item was misprinted or otherwise unavailable for a student to 
respond. The “Not Administered” code was used in the following cases: 

• Achievement item not assigned to the student — All students participating in PIRLS 2011 
received only one of the 13 test booklets; students in prePIRLS 2011 received one of 12 booklets. 
All variables corresponding to items that were not present in a student’s assigned booklet were 
coded as “Not Administered.” 

• Student absent from session — When a student was not present for a particular testing session, 
all variables relevant to that session were coded as “Not Administered.” 

• Item left out or misprinted — When a particular item (or a whole page) was misprinted or 
otherwise not available to the student, the corresponding variable was coded as “Not 
Administered.” 
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• Item deleted or mistranslated — An item identified during translation verification or item review 
as having a translation error such that the nature of the question was altered, or as having poor 
psychometric properties, was coded as “Not Administered.” 

Not Reached Response Codes (SAS: .R ; SPSS: 6) 

An item was considered not reached when, within passage 1 or passage 2 of a booklet, the item itself 
and the item immediately preceding it were not answered, and there were no other items completed 
in the remainder of that passage. For most purposes, PIRLS and prePIRLS 2011 treated the not- 
reached items as incorrect responses, except during the item calibration step of the IRT scaling, 
when not-reached items were considered to have not been administered (see the scaling section in 
Methods and Procedures in TIMSS and PIRLS 2011). 

PIRLS/prePIRLS Achievement Scores 

Achievement scales were produced for overall reading achievement, its two purposes for reading, 
and its two processes of comprehension, as shown in Exhibit 4.4. A total of five achievement scales 
were produced in PIRLS 2011 and in prePIRLS 201 1. A detailed description of the PIRLS and 
prePIRLS 2011 scaling approaches and how these achievement scales were created is available in the 
scaling section of Methods and Procedures in TIMSS and PIRLS 2011. For each achievement scale, 
the PIRLS 2011 database provides five separate estimates of each student’s score on that scale. The 
five estimated scores are known as “plausible values,” and the variability between them encapsulates 
the uncertainty inherent in the scale estimation process. 


Exhibit 4.4: PIRLS/prePIRLS 201 1 Achievement Scales 


PIRLS/prePIRLS 2011 Achievement Scales 


Reading 

REA 

Overall Reading 


Purposes for Reading 

INF 

Reading to Acquire and Use Information 

PIRLS 

LIT 

Reading for Literary Experience 


Processes of 

RSI 

Retrieving and Straightforward Inferencing 


Comprehension 

HE 

Interpreting, Integrating, and Evaluating 


Reading 

REA 

Overall Reading 


Purposes for Reading 

INF 

Reading to Acquire and Use Information 

prePIRLS 

LIT 

Reading for Literary Experience 


Processes of 

RSI 

Retrieving 


Comprehension 

HE 

Inferencing and Integrating 


The plausible values for any given scale are the best available measures of student achievement 
on that scale in the PIRLS 2011 International Database, and should be used as the outcome measure 
in any study of student achievement. Plausible values can be readily analyzed using the IEA 
International Database (IDB) Analyzer (2012) and the SAS programs described in this User Guide. 
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The achievement score variable names are based on an eight-character alphanumeric code 
(e.g., ASRINF01), which adheres to the following rules: 

• The first character is always “A,” corresponding to a fourth grade score; 

• The second character is always “S,” to indicate it is a student score variable; 

• The third character is always “R,” to indicate a reading score, whether it is an overall score, a 
purpose score, or process score; 

• The fourth through sixth characters are a three-character code describing the achievement scale, 
as shown in Exhibit 4.4; and 

• The seventh and eighth characters are a two-digit number indicating the plausible value — ”01,” 
“02,” “03,” “04,” or “05.” 

For example, ASRINF01 is the first plausible value on the PIRLS or prePIRLS informational 
purpose achievement scale. 

PIRLS International Benchmarks of Achievement 

To help users of the PIRLS 2011 achievement results understand what performance on the overall 
reading achievement scale signifies in terms of the types of items students could answer successfully, 
PIRLS identified four points on the overall reading scale to serve as International Benchmarks. As 
shown in Exhibit 4.5, the PIRLS International Benchmark scores are 625, 550, 475, and 400, which 
correspond to the Advanced International Benchmark, the High International Benchmark, the 
Intermediate International Benchmark, and the Low International Benchmark, respectively. PIRLS 
used a technique known as scale anchoring to summarize and describe student achievement at these 
four points on the scale. 4 The PIRLS 2011 International Results in Reading presents the results of 
this scale anchoring, and reports the percentage of students in each country reaching each of the 
PIRLS International Benchmarks. 

Exhibit 4.5: PIRLS 201 1 International Benchmarks of Reading Achievement 


Scale Score 

International Benchmark 

625 

Advanced International Benchmark 

550 

High International Benchmark 

475 

Intermediate International Benchmark 

400 

Low International Benchmark 


To assist analysts in using the PIRLS International Benchmarks in secondary analyses, the 
PIRLS 2011 International Database contains a set of variables indicating which International 
Benchmark the students reached. There is a benchmark variable for each plausible value of the 
overall reading scale. The International Benchmark variables follow the achievement score variable 
naming convention, where the fourth through sixth positions have the letters “IBM.” Thus, 


4 The scale anchoring procedure is described in the scale anchoring section of the Methods and Procedures in TIMSS and PIRLS 2011. 
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ASRIBM01-05 are the five benchmark variables for overall reading. The codes used for the 
benchmark variables are described in Exhibit 4.6. 


Exhibit 4.6: 

PIRLS 2011 International Benchmark Variable Codes 

Code 

Description 

1 

Student performed below the Low International Benchmark 

2 

Student performed at or above the Low International Benchmark, but 
below the Intermediate International Benchmark 

3 

Student performed at or above the Intermediate International 
Benchmark, but below the High International Benchmark 

4 

Student performed at or above the High International Benchmark, but 
below the Advanced International Benchmark 

5 

Student performed at or above the Advanced International Benchmark 


4.4.2 PIRLS/prePIRLS 201 1 Within-country Scoring Reliability Data Files (ASR) 

The PIRLS and prePIRLS 2011 within-country scoring reliability data files contain data that can be 
used to investigate the reliability of the PIRLS and prePIRLS constructed response item scoring. The 
scoring reliability data files contain one record for each assessment booklet that was double scored 
during the within-country scoring reliability exercise (see Operations and Quality Assurance in 
Methods and Procedures in TIMSS and PIRLS 2011). For each constructed response item in the 
PIRLS and prePIRLS assessments, the following three variables are included in the scoring 
reliability data files: 

• Original Score (score assigned by the first scorer); 

• Second Score (score assigned by the second scorer); and 

• Score Agreement (agreement between the two scorers). 

It should be noted that the second score data were used only to evaluate within-country scoring 
reliability and were not used in computing the achievement scores included in the International 
Database and presented in PIRLS 2011 International Results in Reading. 

Scoring Reliability Variable Naming Convention 

The variable names for the Original Score, Second Score, and Score Agreement variables are based 
on the same naming convention as that for the achievement item variables discussed above. The third 
character in the variable name differentiates between the three reliability variables: 

• The Original Score variable has the number “1” as the third character, in accordance with the 
achievement item naming convention (e.g., R31M17CA); 

• The Second Score variable has the letter “R” as the third character (e.g., R3RM17CA); and 

• The Score Agreement variable has the letter “I” as the third character (e.g., R3IM17CA). 
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Scoring Reliability Variable Score Values 

The values contained in both the Original Score and Second Score variables are the scores assigned 
using the PIRLS and prePIRLS scoring guides. The Score Agreement variable may have one of two 
values, denoting either agreement or disagreement between the two scorers, as described in 
Exhibit 4.7. 

Exhibit 4.7: PIRLS/prePIRLS 201 1 Score Agreement Variable Codes 


Code 

Description 

1 

Agreement— Identical scores from both scorers 

0 

Disagreement— Different scores from both scorers 


In general, the data in the Original Score variables are identical to those contained in the student 
achievement data files. In some cases, however, the response scores for specific items were recoded 
after a review of the international item statistics revealed inconsistencies in the original scoring 
guides, or showed that the original scores were not functioning as desired. The recoded score values 
were used in computing the achievement scores reflected in the International Report and are present 
in the student achievement data files. In contrast, the Original Score variables in the scoring 
reliability data files contain the original unrecoded response scores. This was done so that the scoring 
reliability measures indicated in the Score Agreement variables were based on the original scoring 
guides used during the constructed response scoring sessions conducted in each country. 

4.4.3 PIRLS/prePIRLS 201 1 Background Questionnaire Data Files 

There are six usual types of PIRLS 201 1 background questionnaire data files: five data files (student, 
home, teacher, school, and curriculum) correspond to the five types of background questionnaires 
administered in PIRLS and prePIRLS 2011, and the sixth links the student and teacher background 
data. The first five types of data files contain the responses to the questions asked in their respective 
background questionnaires. 

PIRLS/prePIRLS 2011 Student Background Data Files (ASG) 

Students who participated in PIRLS 2011 and prePIRLS 2011 were administered a background 
questionnaire with questions related to their home background, school experiences, and attitudes 
toward reading. The student background data files contain students’ responses to these questions. 
They also contain students’ reading achievement scores (plausible values) to facilitate analyses of 
relationships between student background characteristics and achievement. 

The student background data files also contain a number of identification variables, tracking 
variables, sampling and weighting variables, and derived variables that were used for producing 
exhibits in the international reports. These variables are described later in this chapter. 

PIRLS/prePIRLS 2011 Home Background Data Files (ASH) 

Countries that participated in the PIRLS 2011 and prePIRLS 2011 assessments administered a 
Learning to Read survey, which was completed by the students’ parents or guardians. This survey 
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usually is called the Home Questionnaire. It asked questions about preparations for primary 
schooling, including attendance in preschool and literacy-centered activities in the home before the 
child began school, such as reading books, singing songs, or writing letters or words. Parents 
answered questions about home resources in addition to information about their highest level of 
education and employment situations. 

PIRLS/prePIRLS 2011 Teacher Background Data Files (ATG) 

The teachers of the students who were sampled for PIRLS 201 1 and prePIRLS 201 1 were 
administered a questionnaire with questions pertaining to their background and their teaching 
practices in the classes of the sampled students. Each teacher was asked to respond to a questionnaire 
for each class taught that contained sampled students. The teacher background data files contain one 
record for each teacher and class combination. 

In the teacher background data files, each teacher has a unique identification number 
(IDTEACH) and a link number (IDLINK) that is specific to the class taught by the teacher and to 
which the information in the data record corresponds. The IDTEACH and IDLINK combination 
uniquely identifies, within a country, a teacher teaching a specific class. Thus, students linked to 
teachers identified by the same IDTEACH but different IDLINK are taught by the same teacher but 
in different classes. The teacher background data files cannot be merged directly with the student 
data files, and they do not contain sampling and weighting information, nor achievement scores. 

It is important to note that the teachers in the teacher background data files do not constitute a 
representative sample of teachers in a country, but rather are the teachers who taught a representative 
sample of students. The teacher data, therefore, should be thought of as attributes of the students to 
which they are linked, and should be analyzed only in conjunction with the student-teacher linkage 
data files. Chapters 2 and 3 of this User Guide describe student-level analyses combining the teacher 
data and the student-teacher linkage data files with the IEA IDB Analyzer software, as well as with 
SAS programs. 

PIRLS/prePIRLS 2011 School Background Data Files (ACG) 

The school background data files contain school principals’ responses to the questions in the 
PIRLS 2011 school background questionnaires. Although school-level analyses where the schools 
are the units of analysis can be performed, it is preferable to analyze school-level variables as 
attributes of students. To perform student-level analyses with school data, the school background 
data files must be merged with the student background data files using the country and school 
identification variables. Details of the merging procedure with the IEA IDB Analyzer, or SAS 
programs, are described in Chapters 2 and 3 of this User Guide, respectively. 

PIRLS/prePIRLS 2011 Student-Teacher Linkage Data Files (AST) 

The PIRLS and prePIRLS 2011 student-teacher linkage data files contain information required to 
link the student and teacher data files. The student-teacher linkage data files contain one entry per 
student-teacher linkage combination in the data. For instance, if three teachers are linked to a 
student, there are three entries in the file corresponding to that student. The sole purpose of the 
student-teacher linkage data files is to link teacher-level data with student-level data to perform 
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appropriate student-level analyses where teacher characteristics are considered as attributes of the 
students. 

PIRLS 2011 Curriculum Data File 

The PIRLS 2011 curriculum questionnaire data file contains the responses provided by the National 
Research Coordinators of the participating countries to the PIRLS 2011 curriculum questionnaire. 
There is a single PIRLS 201 1 curriculum questionnaire data file because all prePIRLS 2011 
participants also took part in the PIRLS assessment. This file is available in Excel in the PIRLS 201 1 
International Database. 

Background Variable Naming Convention 

The background variable naming convention is based on a 7- or 8-character string. The following 
rules are applied in naming the background variables: 

• The first character is always “A.” 

• The second character indicates the type of respondent. The letter “C” is used to identify data from 
the school principals, the letter “T” is used for teacher data, and the letter “S” for student and 
parent data. 

• The third character is used to indicate the source of the data. The letter “B” is used for all 
background variables associated with questions in the background questionnaires, and the Rasch 
scores to contextual scales constructed from these data. The letter “D” is used for all variables 
derived from responses in the background questionnaires and the index variables derived from 
the background Rasch scores (see the Context Questionnaire Scales in Methods and Procedures 
in TIMSS and PIRLS 2011). 

• The fourth character is used to indicate the subject or topic to which a background question 
refers. The following letters are used: 

G — General questions; 

H — Home questionnaire questions; and 

R — Questions related to reading. 

• The fifth through eighth characters of all background questionnaire variables represent the 
sequential numbering of the questions as presented in their respective questionnaires. 5 

The curriculum questionnaire uses a different variable naming convention. The first three 
characters are either “GEN” for general questions, or “REA” for questions related to reading 
instruction. The remaining characters indicate the sequential location of the questions in the 
curriculum questionnaire. 

Background Variable Location Convention 

Although the background variable naming convention was modified in 201 1 to show explicitly the 
ordering of questions in the background questionnaires, each question also was assigned a unique 


5 Because of the modular structure of the background questionnaires, the sequential numbering can be specific to each module. 
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location code as shown in Exhibit 4.8. This unique code includes the sequence number of the 
question within the questionnaire, the same sequence number now found in the question’s variable 
name. For example, if the location variable is given as SQG-06, it refers to general question 6 in the 
student background questionnaire. This convention is followed in the data almanacs and in the 
description of the variables included in Supplements 1 and 2 to this User Guide. 

Exhibit 4.8: Background Questionnaire Variable Location Convention 


Questionnaire 

Location Variable 

Student Questionnaire 

SQG-*** 

SQR-«** 

General questions 
Reading questions 

Home Questionnaire 

HQ-«** 

All questions 

Teacher Questionnaire 

TQG-### 

TQR-88# 

General questions 
Reading questions 

School Questionnaire 

SCQ-*** 

All questions 

Curriculum Questionnaire 

CQG-*** 

CQR-*** 

General questions 
Reading questions 


••• = sequential numbering of the question location in the questionnaire 


Data Coding Conventions 

A series of conventions also were adopted to code the data included in the background data files. 
This section describes these conventions. 

Background Question Response Code Values 

The values assigned to each of the background variables depend on the item format and the number 
of options available. For categorical questions, sequential numerical values were used to correspond 
to the response options available. The numbers correspond to the sequence of appearance of the 
response options. For example, the first response option was represented with a 1, the second 
response option with a 2, etcetera. Open-ended questions, such as “the number of students in a 
class,” were coded with the actual number given as a response. 

Codes for Missing Values 

A subset of values is reserved for specific codes related to various categories of missing data. 
Omitted Response Codes (SAS: . ; SPSS: 9, 99, 999, ...) 

“Omitted” response codes were used for questions that a student, parent, teacher, or school principal 
should have answered but did not. The length of the omitted response code given to a variable in the 
SPSS data files depends on the number of characters needed to represent the variable. In all cases, 
the space necessary to represent the variable is filled with 9’s. For questionnaire data, no distinction 
was made between items left blank and items with invalid answers, such as checking two or more 
response options in a categorical question, or unreadable or uninterpretable responses to open-ended 
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questions. In a small number of cases, data received from a country in an invalid or inconsistent 
manner also were coded as “Omitted.” 

Not Administered Response Codes (SAS: .A ; SPSS: sysmis) 

Special codes were given to items that were “Not Administered,” to distinguish these cases from data 
that were missing due to non-response. In general, the “Not Administered” code was used when an 
entire questionnaire was not completed or a specific question was not administered, such as when a 
question was left out of the instrument or misprinted. The “Not Administered” code was used in the 
following cases: 

• Background question left out or misprinted — When a particular question (or a whole page) was 
misprinted, or otherwise not available to the respondent, the corresponding variables were coded 
as “Not Administered.” 

• Background questions removed — Variables corresponding to questions in the student, home, 
teacher, or school background questionnaires that were considered not applicable in some 
countries were not included in the national versions of the questionnaires. These questions were 
coded as “Not Administered.” 

• Background questions mistranslated or not internationally comparable — In some cases, questions 
in the international version of the questionnaires were mistranslated or modified to fit the 
national context. Whenever possible, modified questions were recoded to match as closely as 
possible the international version. When this was not possible, modified questions were coded as 
“Not Administered.” 

Not Applicable Response Codes (SAS: .B ; SPSS: 6, 96, 996, ...) 

“Not Applicable” response codes were used for the background questionnaire items for which 
responses were dependent upon a filter question. Generally, a “No” response to a filter question lead 
to any follow-up questions being coded as “Not Applicable” because there were no appropriate 
responses to these follow-up questions. 

Contextual Scales and Derived Variables 

In the PIRLS 2011 questionnaires, often several questions were asked about various aspects of a 
single contextual construct. In these cases, responses to the individual items were combined to create 
a score, using Rasch scaling, which provided a more comprehensive picture of the contextual 
construct of interest than the individual variables could on their own. These Rasch scores also were 
categorized, usually into three groups, to create an index. The Rasch scores and their indices are 
included in the International Database and described in the contextual questionnaire scaling section 
of Methods and Procedures in TIMSS and PIRLS 2011. 

Additional variables were derived from responses to multiple questions to provide more pertinent 
information for reporting. Parents’ education is an example where responses from both parents were 
combined into a single variable in order to report a single educational level. Generally, records 
(whether students, parents, teachers, or schools) were included in the calculation only if data were 
available for at least two-thirds of the variables involved. For example, if a derived variable was 
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based on six component variables, records that were missing responses to more than two of these 
were counted as missing on the derived variable. Supplement 3 to the User Guide provides a 
description of the derived variables included in the International Database. 

Sampling and Weighting Variables 

Several sampling and weighting variables are included in the PIRLS and prePIRLS 201 1 data files. 
They are listed and described in Exhibit 4.9 and Exhibit 4.10 illustrates the location of the various 
sampling and weighting variables among the different types of data files. It is important to note that 
the teacher background data files do not have any sampling and weighting variables, nor do the home 
background data files. 

Exhibit 4.9: PIRLS/prePIRLS 201 1 Sampling and Weighting Variables 


Variable Name 

Description 

JKZONE 

The sampling zone, or stratum, to which the student's school is 
assigned 

JKREP 

The sampling replicate, or primary sampling unit, to which the 
student's school is assigned 

JKCZONE 

The sampling zone, or stratum, to which the school is assigned 

JKCREP 

The sampling replicate, or primary sampling unit, to which the 
school is assigned 

TOTWGT 

Total student weight— sums to the national population 

SENWGT 

Student senate weight— sums to 500 in each country 

HOUWGT 

Student house weight— sums to the national student sample size 

SCHWGT 

School weight 

TCHWGT 

Teacher weight 

WGTFAC1 

School weighting factor 

WGTADJ1 

School weighting adjustment 

WGTFAC2 

Class weighting factor 

WGTADJ2 

Class weighting adjustment 

WGTFAC3 

Student weighting factor 

WGTADJ3 

Student weighting adjustment 


Although TOTWGT has desirable properties, it also has drawbacks for some analyses. Because 
TOTWGT sums to the student population size in each country, analyses using TOTWGT that 
combine countries will have proportionately more students from larger countries and fewer from 
smaller countries, which may not be desirable for some purposes. For cross-country analyses in 
which countries should be treated equally, PIRLS provides SENWGT, a transformation of TOTWGT 
that results in a weighted sample size of 500 in each country. Additionally, because TOTWGT 
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inflates sample sizes to estimate the population size, software systems that use the actual sample size 
to compute significance tests will give misleading results for analyses weighted by TOTWGT. 
HOUWGT, another transformation of TOTWGT, ensures that the weighted sample corresponds to 
the actual sample size in each country. 

Exhibit 4.1 0: Location of Sampling and Weighting Variables in the PIRLS/prePIRLS 201 1 Data Files 


Sampling and 
Weighting 


Data File Type 


Variable 

ASG 

ASA 

AST 

ACG 

JKZONE 

• 

• 

• 


JKREP 

• 

• 

• 


JKCZONE 




• 

JKCREP 




• 

TOTWGT 

• 

• 



SENWGT 

• 

• 



HOUWGT 

• 

• 



SCHWGT 




• 

TCHWGT 



• 


WGTFAC1 

• 



• 

WGTADJ1 

• 



• 

WGTFAC2 

• 




WGTADJ2 

• 




WGTFAC3 

• 




WGTADJ3 

• 





The weight variables TOTWGT, SENWGT, and HOUWGT are designed for use in student-level 
analyses from all student, home, and school background data files. The weight variable SCHWGT is 
designed for use in school-level analyses where the schools are the units of analysis. 

The weight variable TCHWGT is specifically designed for using teacher background data in 
student-level analyses and is based on TOTWGT. The teacher weight is located in the student- 
teacher linkage files (AST), not in the actual teacher background data files (ATG). Analyses with 
teacher data will be properly weighted by merging the teacher files with the student-teacher linkage 
files. 

The sampling variables beginning with the letters “JK” are used to compute standard errors based 
on the jackknife repeated replication methodology. All weighting variables beginning with the letters 
“WGT” are included to provide insight into the multi-stage sampling and weighting methodology 
applied to the PIRLS and prePIRLS data. All weighting variables are described in the sampling 
section of Methods and Procedures in TIMSS and PIRLS 2011. 
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Structure and Design Variables 

Besides the variables used to store responses to the background questionnaires and achievement 
booklets, the PIRLS and prePIRLS 2011 data files also contain variables meant to store information 
that identify and describe the respondents and design information required to properly analyze the 
data. 

Identification Variables 

In all PIRLS and prePIRLS data files, several identification variables are included that provide 
information to identify countries, students, teachers, or schools. These variables also are used to link 
cases between the different data file types. The identification variables have the prefix “ID” and are 
described below. 

IDCNTRY 

IDCNTRY is a five-digit country identification code based on the ISO 3166 classification as shown 
in Exhibit 4.1. This variable should always be used as the first linking variable whenever files are 
linked within and across countries. 

IDPOP 

IDPOP identifies the target grade and is set to “1,” to indicate primary schooling. 

IDGRADE 

IDGRADE identifies the target grade of the participating students. In PIRLS and prePIRLS 2011, the 
usual value is “4” for most countries. 

IDSCHOOL 

IDSCHOOL is a four-digit identification code that uniquely identifies the participating schools 
within each country. The school codes are not unique across countries. Schools across countries can 
be uniquely identified only with the IDCNTRY and IDSCHOOL combination of linking variables. 

IDCLASS 

IDCLASS is a six-digit identification code that uniquely identifies the sampled classrooms within a 
country. The variable IDCLASS has a hierarchical structure and is formed by concatenating the 
IDSCHOOL variable and a two-digit sequential number identifying the sampled classrooms within a 
school. Classrooms can be uniquely identified in the database by the combination of IDCNTRY and 
IDCLASS as linking variables. 

IDSTUD 

IDSTUD is an eight-digit identification code that uniquely identifies each sampled student in a 
country. The variable IDSTUD also has a hierarchical structure and is formed by concatenating the 
IDCLASS variable and a two-digit sequential number identifying all students within each classroom. 
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Students can be uniquely identified in the database by the combination of IDCNTRY and IDSTUD 
as linking variables. 

IDBOOK 

IDBOOK identifies the specific assessment booklet that was administered to each student. In PIRLS, 
the booklets are given a numerical value from “1” through “13.” In prePIRLS, the booklets are 
numbered from “1” to “12.” 

IDSTRATE & IDSTRATI 

IDSTRATE and IDSTRATI are identification variables generated by the school sampling process. 
IDSTRATE identifies the explicit strata and IDSTRATI the implicit strata from which the 
participating schools were sampled. The codes assigned to these two variables vary from country to 
country and are documented in Supplement 4 to the User Guide. 

IDTEACH 

IDTEACH is a six-digit identification code that uniquely identifies a teacher within a school. It has a 
hierarchical structure and is formed by the concatenation of IDSCHOOL and a two-digit sequential 
number within each school. 

IDLINK 

IDLINK uniquely identifies the class for which a teacher answered a questionnaire. The combination 
of linking variables IDCNTRY, IDTEACH, and IDLINK uniquely identifies all teacher-class 
combinations in the database. 

Exhibit 4.1 1 shows in which data files the various identification variables are located. It also 
highlights the combinations of variables used to uniquely identify the records contained in the 
different data file types. In the student background, home background, and achievement data files, 
the variables IDCNTRY and IDSTUD provide a unique identification number to identify all students 
in the database. Because teachers may teach more than one class, the combination of the IDCNTRY, 
IDTEACH, and IDLINK variables in the teacher background data files is needed to uniquely identify 
all teachers and the classes they teach. Teacher background variables are linked to the appropriate 
students using the student-teacher linkage data files. The variable IDSCHOOL, contained in all files, 
is a unique identification number for each school within a country. Combined with IDCNTRY, it can 
be used to link school background data to corresponding students or teachers. 
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Exhibit 4.1 1 : Location of Identification Variables in the PIRLS/prePIRLS 201 1 Data Files 


Identification 



Data File Type 



Variable 

ASA 

ASG 

AST 

ATG 

ACG 

ASH 

IDCNTRY 

• 

• 

• 

• 

• 

• 

IDGRADE 

• 

• 

• 

• 

• 

• 

IDPOP 

• 

• 

• 

• 

• 

• 

IDSCHOOL 

• 

• 

• 

• 

• 

• 

IDCLASS • • • • • 

IDSTUD • • • • 

IDBOOK • • 

IDSTRATE • • 

ID ST RATI • • 

I DTE AC H 



• 

• 



IDLINK 



• 

• 




Tracking Variables 

Information about students, teachers, and schools provided by the survey tracking forms is stored in 
the tracking variables. 6 These variables have the prefix “IT.” All tracking variables are included in 
the student background data files. ITLANG is included in the student achievement data files, student 
background data files, and home background data files. 

ITSEX 

Gender of each student as stated in the Student Tracking Forms. 

ITBIRTHM and ITBIRTHY 

Month and year of birth of each student as stated in the Student Tracking Forms. 

ITDATEM and ITDATEY 

Month and year of testing for each student. 

ITLANG 

Language of testing for each student. It is set to “1” for all countries that tested in a single language. 
For countries that administered the test in more than one language, additional numerical codes are 
used that correspond to the order of the testing languages as shown in Supplement 2 to the User 
Guide. 


6 Survey tracking forms are lists of students, teachers, and schools used for sampling and administrative purposes. 
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4.5 PIRLS/prePIRLS 201 1 Codebook Files 


All information related to the structure of the PIRLS and prePIRLS 2011 data fdes, as well as the 
source, format, descriptive labels, and response option codes for all variables, is contained in 
codebook files. Each data file type in the International Database is accompanied by a codebook file, 
with the exception of the curriculum data files. The naming convention for codebook files is as 
follows: 

• The first three characters of the filename are in every respect identical to those in the file names 
shown in Exhibit 4.2; 

• The next three characters identify the files as PIRLS or prePIRLS codebooks and are always 
“PRL”; 

• The seventh and eighth characters are either “R3,” to indicate the PIRLS 201 1 study cycle, or 
“LI,” to indicate the prePIRLS 2011 study cycle; and 

• The three-character file extension is either .XLS for the Excel format, or .PDF for the printable 
format. 

Codebook files can be read using Excel, or printed directly from PDF format using Acrobat 
Reader. They describe the contents and structure of the PIRLS and prePIRLS data files. Important 
codebook fields include FIELD_LABL, which contains extended textual information for all 
variables, QUEST LOC, which provides the location of questions and achievement items within 
their respective survey instruments, and FIELD CODE, which lists all acceptable responses allowed 
in the database. 

4.6 PIRLS/prePIRLS 201 1 Data Almanac Files 

Data almanacs provide weighted summary statistics for all variables in the PIRLS and prePIRLS 
201 1 data files. There are two basic types of data almanacs: achievement data almanacs for the 
achievement items, and background data almanacs for the background variables. All data almanac 
files are provided in Word format and printable PDF format. 

Achievement Data Almanacs 

The achievement data almanacs provide weighted summary statistics for each participating country 
on each individual achievement item included in the PIRLS 2011 and prePIRLS 201 1 assessments. 
There are separate achievement data almanacs for PIRLS and prePIRLS and by purposes for reading 
(INF and LIT). The achievement data almanacs display for each item its classification in the 
purposes for reading and the processes of comprehension, the passage to which it belongs, a brief 
description of the item, its variable name, whether it is a multiple choice or constructed response 
item, the correct response key if it is a multiple choice item, and its point value if it is a constructed 
response item. The trend item almanacs, available only for PIRLS, provide summary statistics for 
achievement items used in both the 2006 and 201 1 assessments. The achievement data almanac files 
available in the International Database are listed in Exhibit 4.12. 
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Exhibit 4.1 2: PIRLS/prePIRLS 201 1 Achievement Data Almanacs 


Achievement Data Almanac 

Contents 

Pll_INF_ItemAlmanac 

PIRLS Almanac for informational items 

Pll_LIT_ItemAlmanac 

PIRLS Almanac for literary items 

Pll_INF_TrendItemAlmanac 

PIRLS Trend almanac for informational items 

Pll_LIT_TrendItemAlmanac 

PIRLS Trend almanac for literary items 

pPll_INF_ItemAlmanac 

prePIRLS Almanac for informational items 

pPll_LIT_ItemAlmanac 

prePIRLS Almanac for literary items 


The achievement data almanacs also display the international averages for each item, with each 
country weighted equally. The out-of-grade countries and benchmark participants, listed below the 
international averages, are not included in the calculation of international averages. 

There are two types of displays in the achievement data almanacs, depending on whether an item 
is a multiple choice item or a constructed response item. The statistics displayed in these almanacs 
are as follows: 

• N — The number of students to whom the item was administered. 

• DIFF — Percent of students that responded correctly to a multiple choice item. 

• A, B, C, and D — The percent of students choosing each one of the response options for a 
multiple choice item. 

• Scoring Guide Codes (e.g., 0, 1,2, 3) — The percent of student responses assigned each of the 
codes in the scoring guide for a constructed response item. 

• OMITTED — The percent of students that omitted, or did not respond to, the item. 

• NOT REACHED — The percent of students that did not reach the item. 

• VI, V2, and V3 — The percent of students that scored 1 point or better on the item (VI), 2 points 
or better on the item(V2), or 3 points (V3). 

• l.GIRL %RIGHT and 2.BOY %RIGHT — The percent of girls and boys that either got a 
multiple choice item correct, or obtained the maximum score on a constructed response item. 

Background Data Almanacs 

Background data almanac files contain weighted summary statistics for each participating country on 
each variable in the student, home, teacher, and school background questionnaires, including the 
contextual scales and their indices and the derived variables based on these background variables. 
Among the statistics reported is mean reading achievement by response category. The background 
data almanacs also display for each variable the question as it was asked, its location in the 
corresponding questionnaire, and its variable name in the data files. The background data almanac 
files available in the PIRLS 2011 International Database are listed in Exhibit 4.13. 
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Exhibit 4.1 3: PIRLS/prePIRLS 201 1 Background Data Almanacs 


Background Data Almanac 

Contents 

PI l_StudentAlmanac 

PIRLS student background almanac with reading achievement 

Pll_HomeAlnnanac 

PIRLS home background almanac with reading achievement 

Pll_TeacherAlmanac 

PIRLS teacher background almanac with reading achievement 

PI l_SchoolAlmanac 

PIRLS school background almanac with reading achievement 

pPll_StudentAlmanac 

prePIRLS student background almanac with reading achievement 

pPll_HomeAlnnanac 

prePIRLS home background almanac with reading achievement 

pPll_TeacherAlmanac 

prePIRLS teacher background almanac with reading achievement 

pPll_SchoolAlmanac 

prePIRLS school background almanac with reading achievement 


The background data almanacs also display the international averages for each variable, with 
each country weighted equally. The out-of-grade countries and benchmark participants, listed below 
the international averages, are not included in the calculation of international averages. 

There are two types of displays in the background data almanacs, depending on whether the data 
are categorical (i.e., have a small number of discrete values) or continuous. The almanac display for 
categorical variables includes the following: 

• The sample size (number of students, parents, teachers, or schools included in the sample); 

• The number of valid cases (number of students, parents, teachers, or schools for whom valid data 
were obtained); 

• The weighted percentages of students corresponding to each valid response option (percentages 
based only on the students with valid data, as well as “Not Applicable” codes when used); 

• The weighted percentages of students for whom none of the valid response options were selected, 
coded as “Not Administered” or “Omitted” (percentages based on the sample size); 

• The weighted mean achievement values of students corresponding to each valid response option, 
as well as the “Not Administered” and “Omitted” codes; and 

• In cases where a variable can be coded as “Not Applicable” because of an earlier filter question, 
the weighted percentage of students for whom the variable is coded as “Not Applicable” also is 
displayed, along with the corresponding weighted mean achievement. 

The almanac display for continuous variables includes the following: 

• The sample size (number of students, parents, teachers, or schools included in the sample); 

• The number of valid cases (number of students, parents, teachers, or schools for whom valid data 
were obtained); 

• The weighted percentages of students for whom the variable is coded as “Not Administered” or 
“Omitted” (percentages based on the sample size); 
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• The weighted mean, mode, minimum, maximum, and the 5th, 10th, 25th, 50th, 75th, 90th, and 
95th percentiles across students (based only on the students with valid data, as well as “Not 
Applicable” codes when used); and 

• In cases where a variable can be coded as “Not Applicable” because of an earlier filter question, 
the weighted percentage of students for whom the variable is coded as “Not Applicable” also is 
displayed. 

It is important to note that all percentages, and distributional statistics of continuous variables, 
reported in the background data almanacs are always based on student-level calculations — for 
example, the percentage of students whose teachers or schools gave a particular response to a 
question, because teacher data and school data usually were analyzed as student attributes. 

4.7 PIRLS/prePIRLS 201 1 Program Files 

The PIRLS 2011 International Database includes a number of SAS programs and macros designed to 
facilitate the manipulation of the PIRLS and prePIRLS 2011 data files and conduct proper statistical 
analyses taking into account the jackknife algorithm and the presence of plausible values. These 
programs are described in Chapter 3 of this User Guide. 

The International Database contains two SPSS syntax files — ASASCRR3.SPS and 
ASASCRL1.SPS — which will recode the responses to the individual items from the PIRLS and 
prePIRLS 2011 achievement data files to their appropriate score levels. The files function much the 
same way as their SAS counterparts described in Chapter 3 of this User Guide (see Section 3.4). The 
International Database also includes a third SPSS syntax file to perform a variable recode required 
for the proper execution of the third example in Chapter 2 of this User Guide (see Section 2.5). 
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Appendix 

Organizations and Individuals 
Responsible for PIRLS 201 1 


PIRLS 2011 was a collaborative effort involving hundreds of individuals around 
the world. This appendix acknowledges the individuals and organizations for 
their contributions. Given that work on PIRLS 2011 has spanned approximately 
five years and has involved so many people and organizations, this list may 
not include all who contributed. Any omission is inadvertent. PIRLS 2011 
also acknowledges the students, parents, teachers, and school principals who 
contributed their time and effort to the study. This report would not be possible 
without them. 


Management and Coordination 

PIRLS is a major undertaking of IEA, and together with the Trends in 
International Mathematics and Science Study (TIMSS) comprises the core 
of IEA’s regular cycles of studies. The PIRLS assessment at the fourth grade 
complements TIMSS, which regularly assesses mathematics and science 
achievement at fourth and eighth grades. 

The TIMSS & PIRLS International Study Center at Boston College has 
responsibility for the overall direction and management of the TIMSS and 
PIRLS projects. Headed by Executive Directors Drs. Ina V.S. Mullis and 
Michael O. Martin, the study center is located in the Lynch School of Education. 
In carrying out the project, the TIMSS & PIRLS International Study Center 
worked closely with the IEA Secretariat in Amsterdam, which managed country 
participation, was responsible for verification of all translations produced by 
the participating countries, and coordinated the school visits by International 
Quality Control Monitors. The IEA Data Processing and Research Center in 
Hamburg was responsible for processing and verifying the data submitted by the 
participants; Statistics Canada in Ottawa was responsible for school and student 
sampling activities; and Educational Testing Service in Princeton, New Jersey 
consulted on psychometric methodology, provided software for scaling the 
achievement data, and replicated the achievement scaling for quality assurance. 
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The Project Management Team, comprising the study directors and 
representatives from the TIMSS & PIRLS International Study Center, IEA 
Secretariat and IEA Data Processing and Research Center, Statistics Canada, 
and ETS met twice a year throughout the study to discuss the study’s progress, 
procedures, and schedule. In addition, the study directors met with members of 
IEA’s Technical Executive Group twice yearly to review technical issues. 

To work with the international team and coordinate within-country 
activities, each participating country designates an individual to be the PIRLS 
National Research Coordinator (NRC). The NRCs have the challenging task 
of implementing the PIRLS study in their countries in accordance with the 
PIRLS guidelines and procedures. In addition, the NRCs provide feedback 
and contributions throughout the development of the PIRLS assessment. The 
quality of the PIRLS assessment and data depends on the work of the NRCs 
and their colleagues in carrying out the complex sampling, data collection, and 
scoring tasks involved. Continuing the tradition of exemplary work established 
in previous cycles of PIRLS, the PIRLS 2011 NRCs performed their many tasks 
with dedication, competence, energy, and goodwill, and have been commended 
by the IEA Secretariat, the TIMSS & PIRLS International Study Center, the 
IEA Data Processing and Research Center, and Statistics Canada for their 
commitment to the project and the high quality of their work. 

Funding 

Funding for PIRLS 2011 was provided primarily by the participating countries. 
The National Center for Education Statistics of the US Department of Education 
was a major funding partner, providing funding under contract number 
ED08C00117. The content of this publication does not necessarily reflect the 
views or policies of the US Department of Education nor does mention of 
trade names, commercial products, or organizations imply endorsement by the 
US Government. 

The financial support from Boston College is gratefully acknowledged, as 
well as that from the UK’s National Foundation for Educational Research. 
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IEA Secretariat 

Seamus Hegarty, IEA Chair 

Hans Wagemaker, Executive Director 

Barbara Malak, Manager, Member Relations 

Juriaan Hartenberg, Financial Manager 

Paulina Korsnakova, Senior Professional Researcher 

Isabelle Braun-Gemin, Financial Manager Assistant 

David Ebbs, Management Assistant 

Alana Yu, Management Assistant 

TIMSS & PIRLS International Study Center at Boston College 

Ina V.S. Mullis, Executive Director 
Michael O. Martin, Executive Director 

Pierre Foy, Director of Sampling, Psychometrics, and Data Analysis 

Paul Connolly, Director, Graphic Design and Publications 

Kathleen T. Drucker, Assistant Research Director, PIRLS Reading 

Alka Arora, Assistant Research Director, TIMSS Mathematics 

Gabrielle M. Stanco, Assistant Research Director, TIMSS Science 

leva Johansone, Assistant Research Director, Operations and Quality Control 

Marcie Bligh, Manager of Office Administration 

Bradley Brossman, Psychometrician (through 2012) 

Courtney E. Castle, Graduate Assistant 

Victoria A.S. Centurino, Graduate Assistant 

Susan Farrell, Senior Data Graphics Specialist 

Joseph Galia, Senior Statistician/ Programmer 

Christine Hoage, Manager of Finance 

Lee R. Jones, Chief Science Consultant 

Chad A. Minnich, Writer/Editor 

Jennifer Moher Sepulveda, Data Graphics Specialist 

Mario A. Pita, Senior Data Graphics Specialist 

Jyothsna Pothana, Statistician/Programmer 

Betty Poulos, Administrative Coordinator 

Corinna Preuschoff, Senior Research Specialist (through 2011 ) 

Moira A. Ragan, Graduate Assistant 
Ruthanne Ryan, Data Graphics Specialist 
Steven A. Simpson, Data Graphics Specialist 
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IEA Data Processing and Research Center 

Dirk Hastedt, Co-Director 

Juliane Hencke, Project Co-Manager, TIMSS and PIRLS data processing 

Oliver Neuschmidt, Project Co-Manager, TIMSS and PIRLS data processing 

Yasin Afana, Deputy Project Manager, TIMSS and PIRLS data processing 

Milena Taneva, Deputy Project Manager, TIMSS and PIRLS data processing 

Alena Becker, Research Analyst 

Christine Busch, Research Analyst 

Ralph Carstens, Senior Research Analyst 

Mark Cockle, Research Analyst 

Tim Daniel, Research Analyst 

Limiao Duan, Research Analyst 

Eugenio Gonzales, Unit Head, Research and Analysis 

Pamela Inostroza, Research Analyst 

Michael Jung, Research Analyst 

Maike Junod, Programmer 

Alexander Konn, Programmer 

Marta Kostek-Drosihn, Unit Coordinator 

Sabine Meinck, Research Analyst, Sampling 

Sebastian Meyer, Research Analyst 

Dirk Oehler, Research Analyst 

Moritz Otto, Programmer 

Devi Potham Rajendra Prasath, Programmer 

Daniel Radtke, Research Analyst 

Anke Sielemann, Research Analyst 

Harpreet Singh Choudry, Unit Head, Software 

Sabine Tieck, Research Analyst 

Caroline Vandenplas, Research Analyst 

Sabine Weber, Research Analyst 

Bettina Wietzorek, Meeting and Seminar Coordinator 

Meng Xue, Programmer 

Olaf Zuehlke, Research Analyst, Sampling 

Statistics Canada 

Marc Joncas, Senior Methodologist 
Sylvie LaRoche, Senior Methodologist 
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Educational Testing Service 

Matthias Von Davier, Research Director 

Edward Kulick, Research Director 

Meng Wu, Associate Psychometrician 

Jonathan Weeks, Associate Research Scientist 

Scott Davis, Data Analysis and Computational Research Specialist 

Yuxin (Christina) Tang, Principal Research Data Analyst 

Zhumei Guo, Senior Research Data Analyst 

Sampling Referee 

Keith Rust, Vice President and Associate Director of the Statistical Group, Westat, Inc. 


PIRLS 201 1 Reading Development Group 

Jan Mejding 

Department of Education 
Aarhus University 

Denmark 

Sue Horner 

Curriculum Division 

Qualifications and Curriculum Authority 

England 

Elinor Saiegh-Haddad 

English Literature and Linguistics Department 
Bar-Ilan University 

Israel 

Fatma Abdulabus Noorani 
Ministry of Education 

Oman 


Galina Zuckerman 
Psychological Institute 
Russian Academy of Education 

Russian Federation 

Elizabeth Pang 

Curriculum Planning & Development Division 
Ministry of Education 

Singapore 

Caroline Liberg 

Department of Curriculum Studies 
Uppsala University 

Sweden 

Elois Scott 

United States 
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PIRLS 201 1 Item Development Task Force 

Kathleen T. Drucker, Assistant Research Director, PIRLS Reading 
Ann Kennedy, PIRLS Coordinator (through 2010) 

Marian Sainsbury, PIRLS Reading Coordinator (NFER) 

Patricia Donahue, PIRLS Reading Consultant (EPS) 

Elois Scott, United States 

Julian Fraillon, Web-based Assessment Coordinator (ACER) 


PIRLS 201 1 Questionnaire Development Group 

Hwawei Ko 

Graduate Institute of Learning and Instruction 
National Central University 

Chinese Taipei 

Jasminka Buljan Culej 

National Center for External Evaluation in 
Education 

Croatia 

Marc Colmant 

Ministere de l’Education Nationale, 

Secretariat General Direction de l’Evaluation, de la 
Prospective et de la Performance 

France 

Knut Schwippert 
Faculty of Education 

General, Intercultural, and International 
Comparative Education 

University of Hamburg 

Germany 

Megan Chamberlain 

Comparative Education Research Unit 

Ministry of Education 

New Zealand 


Ragnar Gees Solheim 

National Center for Reading Education and 
Research 

University of Stavanger 

Norway 

Abdessalem Bouslama 

Office of Student Assessment 

Evaluation Institute, Supreme Education Council 

Qatar 

Sarah Howie 

Center for Evaluation and Assessment (CEA) 
Faculty of Education 
University of Pretoria 

South Africa 

Valena Plisko 

National Center for Education Statistics 
U.S. Department of Education 

United States 
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Australia 
Sue Thomson 

Australian Council for Educational Research 
Austria 

Gunter Haider 
Birgit Suchan 

Bundesinstitut fuer Bildungsforschung, 
Innovation und Entwicklung des 
Oesterreichischen Schulwesens (BIFIE) 

Azerbaijan 

Emin Meherremov 

Ulviya Mikailova (through 2010) 

Department of Monitoring and Assessment 
Ministry of Education 

Belgium (French) 

Genevieve Hindryckx 
Anne Matoul 

Annette Lafontaine (through 2011) 
Universite de Liege 

Botswana 

Chawangwa Mudongo 
Botswana Examinations Council 

Bulgaria 

Marina Vasileva Mavrodieva 

Center for Control & Assessment of 
the Quality in School Education 

Canada 
Pierre Brochu 

Council of Ministers of Education 


Chinese Taipei 
Hwawei Ko 

Graduate Institute of Learning and Instruction 
National Central University 

Colombia 
Francisco Reyes 

Instituto Colombiano para el Fomento de 
la Educacion Superior (ICFES) 

Croatia 

Jasminka Buljan Culej 

National Center for External Evaluation in 
Education 

Czech Republic 
Iveta Kramplova 

Dagmar Pavlikova (through 2009) 

Czech School Inspectorate 

Denmark 
Jan Mejding 

Department of Education 
Aarhus University 

England 
Liz Twist 

National Foundation for Educational Research 

Finland 
Sari Sulkunen 

Finnish Institute for Educational Research 
University of Jyvaskyla 

France 

Marc Colmant 

Ministere de l’Education Nationale, 

Direction de FEvaluation, 

de la Prospective et de la Performance 
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Georgia 

Nutsa Kobakhidze 

National Assessment and Examinations Center 

Germany 
Wilfried Bos 
Irmela Tarelli 

Institute for School Development Research (IFS) 
TU Dortmund University 

Honduras 

Renan Rapalo Castellanos 
Secretaria de Educacion 
Instituto de Investigacion 
Universidad Pedagogica Nacional 

Hong Kong 

Tse Shek-Kam 

Faculty of Education 

The University of Hong Kong 

Hungary 
Ildiko Balazsi 
Peter Balkanyi 
Educational Authority 

Department of Eductional Assessment and 
Evaluation 

Indonesia 
Hari Setiadi 

Nugaan Yulia Wardani (through 2010) 

Center for Educational Assessment 
Ministry of National Education 

Iran, Islamic Republic of 
Abdolazim Karimi 

Research Institute for Education (RIE) 

Ministry of Education 


Ireland 
Eemer Eivers 

Educational Research Centre 
St. Patricks College, Dublin 

Israel 

Inbal Ron-Kaplan 

National Authority for Measurement and 
Evaluation in Education (RAMA) 

Italy 

Fabio Alivernini 

Instituto Nazionale per la Valutazione 
del Sistema Educativo di Instruzione e 
di Formazione (INVALSI) 

Kuwait 

Noriah Alroomi 
Ministry of Education 

Lithuania 

Irina Mackeviciene 
Aiste Elijio (through 201 1 ) 

National Examination Centre 
Ministry of Education and Science 

Malta 

Grace Grima 

Directorate for Quality & Standards 
in Education 

Ministry of Education 
Morocco 

Mohammed Sassi 

Departement de l’Evaluation Nationale 

Centre Nationale de 1‘Evaluation et 
des Examens 

Netherlands 
Andrea Netten 

Expertisecentrum Nederlands 
Radboud University 
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New Zealand 
Megan Chamberlain 
Comparative Education Research Unit 
Ministry of Education 

Northern Ireland 
Patricia Wyers 

Statistics and Research Branch 
Department of Education 

Norway 

Ragnar Gees Solheim 
Victor van Daal 

National Centre for Reading Education and 
Research 

University of Stavanger 
Oman 

Zuwaina Saleh Al-Maskari 
Ministry of Education 

Poland 

Krzysztof Konarzewski 
Polish Academy of Sciences 

Portugal 
Ana Ferreira 

Education Statistics and Planning Office 
Ministry of Education 

Qatar 

Abdessalem Bouslama 

Office of Student Assessment 

Evaluation Institute, Supreme Education Council 

Romania 

Gabriela Noveanu 

Institute for Educational Sciences 

Curriculum Department 


Russian Federation 
Galina Kovaleva 

Institute of Content and Methods of Education 

Center for Evaluating the Quality of 
General Education 

Russian Academy of Education 

Saudi Arabia 
Saleh Alshaya 

International Studies & Testing Center (ISTC) 
Ministry of Education 

Singapore 
Ng Huey Bian 

Research and Evaluation Section 
Ministry of Education 

Elizabeth Pang 

Curriculum Planning and 
Development Division 

Ministry of Education 

Slovak Republic 
Sofia Gallova 

National Institute for Certified Educational 
Measurements 

Slovenia 

Marjeta Doupona Horvat 
Educational Research Institute 

South Africa 
Sarah Howie 
Surette van Staden 
Elsie Venter (through 2010) 

Center for Evaluation and Assessment (CEA) 
University of Pretoria 
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Spain 

David Cervera Olivares 
Jesus Dominguez Castillo (through 2012) 
National Institute of Educational Evaluation 
Ministry of Education, Culture and Sports 

Sweden 
Agnes Tongur 

Marie Eklund (through 2010) 

Swedish National Agency for Education 

Trinidad and Tobago 
Mervyn Sambucharan 

Division of Educational Research and Evaluation 
Ministry of Education 

United Arab Emirates 
Aljawhara Ali A1 Sebaiei 
Nada Abu Baker Husain Ruban 
Assessment Department 
Ministry of Education 

United States 

Sheila Thompson 

Stephen Provasnik (through 201 1 ) 

National Center for Education Statistics 
U.S. Department of Education 


BENCHMARKING PARTICIPANTS 

Alberta, Canada 
Ping Yang 

Learner Assessment Branch 
Alberta Education 

Ontario, Canada 
Michael Kozlow 

Education Quality and Accountability Office 

Quebec, Canada 
Robert Marcotte 

Direction de la sanction des etudes 

Andalusia, Spain 
Sebastian Cardenas 
Students Assessment Service 

Andalusian Agency for the 
Educational Assessment 

Abu Dhabi, United Arab Emirates 

Shaikha Ali A1 Zaabi 

Abu Dhabi Education Council 

Dubai, United Arab Emirates 
Rabaa AlSumaiti 

Knowledge and Human Development Authority 

Florida, United States 
Patrick Gonzalez 

National Center for Education Statistics 
U.S. Department of Education 
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